Oct 30 05:30:54.951871 ip-10-0-136-173 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 05:30:55.336981 ip-10-0-136-173 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 05:30:55.336981 ip-10-0-136-173 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 05:30:55.336981 ip-10-0-136-173 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 05:30:55.336981 ip-10-0-136-173 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 05:30:55.340221 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340140    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:30:55.340221 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340222    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340226    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340229    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340232    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340234    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340237    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340240    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340242    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340245    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340247    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340250    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340252    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340255    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340257    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340259    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340262    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340264    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340266    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340269    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:30:55.340288 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340272    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340274    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340276    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340279    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340281    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340284    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340286    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340292    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340295    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340298    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340300    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340303    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340305    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340308    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340310    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340312    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340315    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340317    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340319    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340322    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:30:55.340728 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340324    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340326    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340329    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340331    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340333    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340336    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340340    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340344    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340347    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340350    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340353    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340357    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340360    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340363    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340365    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340368    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340370    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340373    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340375    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:30:55.341174 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340377    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340381    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340383    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340386    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340388    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340390    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340393    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340395    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340398    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340400    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340402    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340404    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340407    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340409    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340411    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340414    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340416    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340419    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340421    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:30:55.341614 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340424    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340426    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340429    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340431    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340433    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340436    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340438    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340441    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340444    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340447    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340449    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340451    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340454    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340456    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340459    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340462    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340464    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340467    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340469    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340473    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:30:55.342076 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340476    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340479    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340481    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340484    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340487    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340490    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340493    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.340496    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341271    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341278    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341281    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341283    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341286    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341289    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341292    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341294    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341297    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341299    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341303    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341306    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:30:55.342532 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341308    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341311    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341313    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341315    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341318    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341320    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341323    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341325    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341333    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341336    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341338    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341341    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341343    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341345    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341348    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341351    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341353    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341356    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341358    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341361    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:30:55.343018 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341363    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341366    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341368    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341370    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341373    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341376    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341381    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341384    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341386    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341389    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341392    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341395    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341397    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341399    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341402    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341404    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341407    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341409    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341411    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:30:55.343489 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341414    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341416    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341424    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341426    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341429    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341431    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341433    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341436    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341438    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341440    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341443    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341445    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341448    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341454    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341459    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341463    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341465    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341468    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341470    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:30:55.343940 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341473    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341475    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341477    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341480    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341482    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341485    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341487    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341489    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341492    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341494    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341497    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341499    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341501    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341504    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341506    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341509    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341517    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341520    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341522    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341524    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:30:55.344370 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341527    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341529    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341532    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341535    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341537    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341540    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341544    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341546    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341548    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341551    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341553    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341555    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341558    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341560    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341562    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.341565    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342139    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342146    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342149    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342152    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:30:55.344850 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342155    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342157    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342160    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342163    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342165    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342168    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342171    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342173    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342177    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342180    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342182    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342185    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342188    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342190    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342193    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342195    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342197    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342200    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342204    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342207    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:30:55.345295 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342209    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342212    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342214    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342216    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342219    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342222    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342225    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342228    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342231    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342233    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342236    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342238    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342241    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342243    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342246    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342249    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342251    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342255    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342258    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:30:55.345765 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342262    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342265    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342268    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342271    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342273    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342276    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342278    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342281    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342283    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342285    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342288    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342292    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342296    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342298    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342301    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342303    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342306    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342308    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342312    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:30:55.346258 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342316    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342319    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342322    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342325    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342327    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342329    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342332    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342334    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342337    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342339    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342342    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342345    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342347    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342350    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342352    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342354    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342363    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342366    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342368    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:30:55.346707 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342371    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342373    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342376    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342378    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342380    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342384    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342388    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342390    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342393    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342395    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342398    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342400    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342403    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342405    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342408    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342410    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342413    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342415    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342417    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342420    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:30:55.347161 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342422    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342424    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342427    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342430    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.342433    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342552    2580 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342559    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342568    2580 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342572    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342577    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342599    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342605    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342609    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342612    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342615    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342619    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342622    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342626    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342630    2580 flags.go:64] FLAG: --cgroup-root=""
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342633    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342636    2580 flags.go:64] FLAG: --client-ca-file=""
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342639    2580 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342642    2580 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 05:30:55.347665 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342649    2580 flags.go:64] FLAG: --cluster-domain=""
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342652    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342655    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342658    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342662    2580 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342665    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342669    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342672    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342675    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342678    2580 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342680    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342683    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342686    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342689    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342694    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342697    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342699    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342702    2580 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342705    2580 flags.go:64] FLAG: --enable-server="true"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342708    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342721    2580 flags.go:64] FLAG: --event-burst="100"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342725    2580 flags.go:64] FLAG: --event-qps="50"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342727    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342730    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 05:30:55.348180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342733    2580 flags.go:64] FLAG: --eviction-hard=""
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342738    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342742    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342746    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342749    2580 flags.go:64] FLAG: --eviction-soft=""
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342752    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342755    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342758    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342761    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342764    2580 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342766    2580 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342769    2580 flags.go:64] FLAG: --feature-gates=""
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342773    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342776    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342779    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342782    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342786    2580 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342789    2580 flags.go:64] FLAG: --help="false"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342791    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342795    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342798    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342800    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342804    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 05:30:55.348757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342807    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342810    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342813    2580 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342816    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342819    2580 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342822    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342832    2580 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342835    2580 flags.go:64] FLAG: --kube-reserved=""
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342838    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342841    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342844    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342848    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342852    2580 flags.go:64] FLAG: --lock-file=""
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342855    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342857    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342860    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342866    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342869    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342872    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342874    2580 flags.go:64] FLAG: --logging-format="text"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342877    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342881    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342884    2580 flags.go:64] FLAG: --manifest-url=""
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342886    2580 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342891    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 05:30:55.349272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342894    2580 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342898    2580 flags.go:64] FLAG: --max-pods="110"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342901    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342904    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342907    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342910    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342912    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342915    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342918    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342925    2580 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342928    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342931    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342934    2580 flags.go:64] FLAG: --pod-cidr=""
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342936    2580 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342945    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342948    2580 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342951    2580 flags.go:64] FLAG: --port="10250"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342954    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342959    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-098a57ab22afcd504"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342963    2580 flags.go:64] FLAG: --qos-reserved=""
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342966    2580 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342969    2580 flags.go:64] FLAG: --register-node="true"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342972    2580 flags.go:64] FLAG: --register-with-taints=""
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342976    2580 flags.go:64] FLAG: --registry-burst="10"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342979    2580 flags.go:64] FLAG: --registry-qps="5"
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342981    2580 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 05:30:55.349846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342984    2580 flags.go:64] FLAG: --reserved-memory=""
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342988    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342991    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342994    2580 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.342997    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343000    2580 flags.go:64] FLAG: --runonce="false"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343003    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343006    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343009    2580 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343011    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343014    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343017    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343020    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343023    2580 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343026    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343029    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343032    2580 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343041    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343044    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343047    2580 flags.go:64] FLAG: --system-cgroups=""
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343050    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343060    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343063    2580 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343066    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343074    2580 flags.go:64] FLAG: --tls-min-version=""
Oct 30 05:30:55.350474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343078    2580 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343081    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343084    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343087    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343090    2580 flags.go:64] FLAG: --v="2"
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343094    2580 flags.go:64] FLAG: --version="false"
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343112    2580 flags.go:64] FLAG: --vmodule=""
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343116    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343119    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343336    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343340    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343343    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343346    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343348    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343351    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343353    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343356    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343359    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343361    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343364    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343366    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343370    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:30:55.351092 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343374    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343377    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343380    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343382    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343384    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343388    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343392    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343400    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343403    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343408    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343412    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343414    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343417    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343419    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343421    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343424    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343426    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343429    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343431    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:30:55.351609 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343434    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343436    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343439    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343441    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343443    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343446    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343448    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343450    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343453    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343455    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343458    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343460    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343463    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343465    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343467    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343470    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343472    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343474    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343477    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:30:55.352045 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343479    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343481    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343489    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343494    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343497    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343500    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343502    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343505    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343507    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343509    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343512    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343514    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343516    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343519    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343521    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343524    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343526    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343529    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343531    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:30:55.352478 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343534    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343536    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343538    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343541    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343543    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343546    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343548    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343550    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343553    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343555    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343558    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343560    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343562    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343565    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343567    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343569    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343578    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343582    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343598    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343601    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:30:55.353009 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343604    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343606    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343609    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343611    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343614    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343616    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343618    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343621    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343623    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343625    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343628    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343630    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343632    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343635    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343637    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343640    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:30:55.353469 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.343665    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343712    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343716    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343719    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343722    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343724    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343727    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343729    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343732    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343734    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343736    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343739    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343741    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343754    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343757    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343760    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343762    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343765    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343767    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:30:55.353848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343770    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343772    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343774    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343777    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343780    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343782    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343784    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343787    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343789    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343791    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343794    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343796    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343798    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343801    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343803    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343806    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343808    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343810    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343812    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343815    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:30:55.354316 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343817    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343819    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343822    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343824    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343827    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343829    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343833    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343841    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343844    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343846    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343849    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343851    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343853    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343855    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343858    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343860    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343862    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343865    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343867    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343869    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:30:55.354800 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343872    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343874    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343876    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343879    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343881    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343883    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343886    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343888    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343890    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343893    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343895    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343898    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343900    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343903    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343906    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343910    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343912    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343915    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343919    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343921    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:30:55.355273 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343930    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343934    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343937    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343940    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343942    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343945    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343947    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343950    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343952    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343955    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343957    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343959    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343961    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343964    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343966    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343968    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343971    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343973    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343976    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343978    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:30:55.355734 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343980    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343982    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343985    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343988    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343991    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343995    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.343997    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.344000    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.350916    2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.350931    2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.350948    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.350952    2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351023    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351028    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351031    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351034    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351037    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351039    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351042    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:30:55.356211 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351045    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351047    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351050    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351052    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351055    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351059    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351063    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351066    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351069    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351071    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351074    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351076    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351079    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351081    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351084    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351086    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351089    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351093    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351097    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:30:55.356679 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351100    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351103    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351118    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351122    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351124    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351127    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351129    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351132    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351135    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351137    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351140    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351142    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351145    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351147    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351149    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351152    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351154    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351156    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351159    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351161    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:30:55.357098 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351163    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351166    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351168    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351170    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351172    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351175    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351177    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351179    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351182    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351184    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351187    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351190    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351192    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351195    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351197    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351199    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351202    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351204    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351207    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351209    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:30:55.357570 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351212    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351214    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351217    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351219    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351222    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351224    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351226    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351229    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351231    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351233    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351235    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351238    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351240    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351242    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351244    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351247    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351250    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351252    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351254    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351257    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:30:55.358328 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351259    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351261    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351264    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351266    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351268    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351271    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351273    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351276    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351278    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351280    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351283    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351285    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351287    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351290    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351293    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351295    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351298    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351300    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351303    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:30:55.358819 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351305    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.351330    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351464    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351468    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351471    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351473    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351476    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351478    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351481    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351483    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351485    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351488    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351490    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351492    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351495    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351497    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:30:55.359390 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351500    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351502    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351504    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351506    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351509    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351511    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351514    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351516    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351519    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351521    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351523    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351526    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351528    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351531    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351533    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351535    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351538    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351540    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351542    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351544    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:30:55.359974 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351547    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351549    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351551    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351554    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351556    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351558    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351561    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351563    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351566    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351568    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351571    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351573    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351575    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351577    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351580    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351582    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351602    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351605    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351607    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351610    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:30:55.360491 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351612    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351615    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351618    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351622    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351627    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351630    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351633    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351636    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351638    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351640    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351643    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351646    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351648    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351650    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351653    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351655    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351658    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351660    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351664    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:30:55.360964 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351667    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351669    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351672    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351674    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351676    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351679    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351681    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351684    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351686    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351688    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351691    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351693    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351695    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351698    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351700    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351702    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351705    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351707    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351710    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351713    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:30:55.361397 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351715    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351717    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351720    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351722    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351725    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351727    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351729    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351733    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351736    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351739    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351742    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351747    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351751    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.351773    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351806    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:30:55.361971 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351810    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351813    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351816    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351818    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351821    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351823    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351827    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351830    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351832    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351835    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351837    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351840    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351843    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351845    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351847    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351850    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351853    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351855    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351858    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:30:55.362371 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351860    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351863    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351865    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351867    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351870    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351872    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351875    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351877    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351879    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351881    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351888    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351891    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351893    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351896    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351898    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351901    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351903    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351905    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351908    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:30:55.362842 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351910    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351913    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351915    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351917    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351919    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351922    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351924    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351926    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351929    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351931    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351934    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351937    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351939    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351941    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351944    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351946    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351949    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351951    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351953    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351956    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:30:55.363306 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351958    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351960    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351963    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351969    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351971    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351973    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351976    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351978    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351980    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351983    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351985    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351989    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351992    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351996    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.351998    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352001    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352004    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352006    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352009    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:30:55.363768 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352011    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352013    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352016    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352018    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352020    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352023    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352026    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352028    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352030    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352033    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352035    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352037    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352040    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352042    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352044    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352046    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352051    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352053    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352056    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352058    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:30:55.364186 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352061    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352063    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352065    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352068    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352070    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352074    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352076    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:55.352079    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.352127    2580 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.354128    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.355297    2580 server.go:1014] "Starting client certificate rotation"
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.355392    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.356141    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.357619    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 05:30:55.364846 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.362949    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 05:30:55.371690 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.371670    2580 log.go:25] "Validated CRI v1 runtime API"
Oct 30 05:30:55.374243 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.374227    2580 log.go:25] "Validated CRI v1 image API"
Oct 30 05:30:55.375016 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.374990    2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 05:30:55.377500 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.377481    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9f603ae9-a2e0-42a9-a511-0cc6f2cf44ca:/dev/nvme0n1p3 d5daec70-9968-46eb-98d8-08897dc79571:/dev/nvme0n1p4]
Oct 30 05:30:55.377552 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.377500    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 05:30:55.382856 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.382742    2580 manager.go:217] Machine: {Timestamp:2026-10-30 05:30:55.381293179 +0000 UTC m=+0.319667549 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3204619 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2823b3e9408d8f6a0bf9d2e392ec18 SystemUUID:ec2823b3-e940-8d8f-6a0b-f9d2e392ec18 BootID:89340346-a628-40d7-99f1-fbcfd93ab3f1 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ed:0b:9a:87 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ed:0b:9a:87 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:6d:98:2c:4f:34 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 05:30:55.383441 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.383428    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 05:30:55.383536 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.383518    2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 05:30:55.385247 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.385222    2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 05:30:55.385443 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.385250    2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-173.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 05:30:55.385489 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.385446    2580 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 05:30:55.385489 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.385454    2580 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 05:30:55.385489 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.385478    2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 05:30:55.385562 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.385496    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 05:30:55.386272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.386261    2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 05:30:55.389920 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.389908    2580 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 05:30:55.390062 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.390053    2580 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 05:30:55.390760 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.390750    2580 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 05:30:55.390798 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.390764    2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 05:30:55.390798 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.390778    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 05:30:55.390798 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.390787    2580 kubelet.go:405] "Adding apiserver pod source"
Oct 30 05:30:55.390962 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.390797    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 05:30:55.391126 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.391113    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 05:30:55.393215 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.393198    2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 05:30:55.397853 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.397840    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 05:30:55.398621 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.398609    2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 05:30:55.398653 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.398639    2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 05:30:55.398795 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.398784    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 05:30:55.398824 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.398804    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 05:30:55.398824 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.398815    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 05:30:55.398875 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.398825    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 05:30:55.398875 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.398833    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 05:30:55.398875 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.398846    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 05:30:55.399437 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.399428    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 05:30:55.399469 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.399438    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 05:30:55.399469 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.399450    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 05:30:55.399469 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.399456    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 05:30:55.399469 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.399469    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 05:30:55.399576 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.399478    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 05:30:55.399955 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.399946    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 05:30:55.399989 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.399958    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 05:30:55.404052 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.404031    2580 server.go:1263] "Started kubelet"
Oct 30 05:30:55.404144 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.404084    2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 05:30:55.404357 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.404269    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 05:30:55.404422 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.404413    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 05:30:55.408277 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.408247    2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 05:30:55.408695 ip-10-0-136-173 systemd[1]: Started Kubernetes Kubelet.
Oct 30 05:30:55.411473 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.411220    2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 05:30:55.412520 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.412504    2580 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 05:30:55.416563 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.416546    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 05:30:55.417041 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.417027    2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 05:30:55.417177 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.417153    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:55.417774 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.417753    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 05:30:55.418018 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.418003    2580 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 05:30:55.418018 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.418019    2580 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 05:30:55.418136 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.418117    2580 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 05:30:55.418136 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.418127    2580 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 05:30:55.418320 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.418273    2580 factory.go:153] Registering CRI-O factory
Oct 30 05:30:55.418320 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.418290    2580 factory.go:223] Registration of the crio container factory successfully
Oct 30 05:30:55.418424 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.418340    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 05:30:55.418424 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.418351    2580 factory.go:55] Registering systemd factory
Oct 30 05:30:55.418424 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.418359    2580 factory.go:223] Registration of the systemd container factory successfully
Oct 30 05:30:55.418424 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.418377    2580 factory.go:103] Registering Raw factory
Oct 30 05:30:55.418424 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.418400    2580 manager.go:1199] Started watching for new ooms in manager
Oct 30 05:30:55.420032 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.420016    2580 manager.go:319] Starting recovery of all containers
Oct 30 05:30:55.429648 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.429631    2580 manager.go:324] Recovery completed
Oct 30 05:30:55.433606 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.433578    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 05:30:55.435404 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.435385    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 05:30:55.435477 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.435421    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 05:30:55.435477 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.435437    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientPID"
Oct 30 05:30:55.436031 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.436017    2580 cpu_manager.go:226] "Starting" policy="none"
Oct 30 05:30:55.436031 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.436029    2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 05:30:55.436157 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.436064    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 05:30:55.436157 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.436074    2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 05:30:55.437008 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.436987    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 05:30:55.437092 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.437022    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 05:30:55.437092 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.437036    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientPID"
Oct 30 05:30:55.437477 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.437427    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 05:30:55.438181 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.438156    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-173.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 05:30:55.438279 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.438252    2580 policy_none.go:50] "Start"
Oct 30 05:30:55.438279 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.438267    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 05:30:55.438365 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.438281    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 05:30:55.438410 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.437416    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-173.ec2.internal.18e337a44b772acc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-173.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-173.ec2.internal,},FirstTimestamp:2026-10-30 05:30:55.40385454 +0000 UTC m=+0.342228908,LastTimestamp:2026-10-30 05:30:55.40385454 +0000 UTC m=+0.342228908,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-173.ec2.internal,}"
Oct 30 05:30:55.440799 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.440782    2580 policy_none.go:44] "Start"
Oct 30 05:30:55.457545 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.457516    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 05:30:55.457820 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.457795    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 05:30:55.457872 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.457856    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 05:30:55.468150 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.467993    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 05:30:55.470977 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.470906    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-173.ec2.internal.18e337a44d5894cb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-173.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-173.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-173.ec2.internal,},FirstTimestamp:2026-10-30 05:30:55.435404491 +0000 UTC m=+0.373778870,LastTimestamp:2026-10-30 05:30:55.435404491 +0000 UTC m=+0.373778870,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-173.ec2.internal,}"
Oct 30 05:30:55.482255 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.482222    2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 05:30:55.484572 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.484504    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-173.ec2.internal.18e337a44d58f64e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-173.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-173.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-173.ec2.internal,},FirstTimestamp:2026-10-30 05:30:55.435429454 +0000 UTC m=+0.373803830,LastTimestamp:2026-10-30 05:30:55.435429454 +0000 UTC m=+0.373803830,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-173.ec2.internal,}"
Oct 30 05:30:55.486164 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.486151    2580 manager.go:341] "Starting Device Plugin manager"
Oct 30 05:30:55.486241 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.486177    2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 05:30:55.486241 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.486186    2580 server.go:87] "Starting device plugin registration server"
Oct 30 05:30:55.486435 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.486425    2580 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 05:30:55.486487 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.486437    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 05:30:55.486563 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.486549    2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 05:30:55.486691 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.486674    2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 05:30:55.486691 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.486687    2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 05:30:55.487247 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.487228    2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 05:30:55.487323 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.487279    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:55.492077 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.492063    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-snclq"
Oct 30 05:30:55.513410 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.505176    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-snclq"
Oct 30 05:30:55.522129 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.522061    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-173.ec2.internal.18e337a44d592933  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-173.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-173.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-173.ec2.internal,},FirstTimestamp:2026-10-30 05:30:55.435442483 +0000 UTC m=+0.373816860,LastTimestamp:2026-10-30 05:30:55.435442483 +0000 UTC m=+0.373816860,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-173.ec2.internal,}"
Oct 30 05:30:55.535959 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.535931    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 05:30:55.537299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.537279    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 05:30:55.537369 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.537304    2580 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 05:30:55.537369 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.537329    2580 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 05:30:55.537369 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.537359    2580 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 05:30:55.564872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.564850    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 05:30:55.587381 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.587334    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 05:30:55.589152 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.589136    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 05:30:55.589218 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.589167    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 05:30:55.589218 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.589177    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientPID"
Oct 30 05:30:55.589218 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.589200    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.607315 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.607296    2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.607368 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.607319    2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-173.ec2.internal\": node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:55.638346 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.638314    2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"]
Oct 30 05:30:55.638447 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.638368    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 05:30:55.639175 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.639160    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 05:30:55.639254 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.639184    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 05:30:55.639254 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.639196    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientPID"
Oct 30 05:30:55.640940 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.640927    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 05:30:55.640992 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.640938    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:55.641115 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.641098    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.641170 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.641135    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 05:30:55.641659 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.641646    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 05:30:55.641729 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.641659    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 05:30:55.641729 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.641669    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 05:30:55.641729 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.641676    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 05:30:55.641729 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.641680    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientPID"
Oct 30 05:30:55.641729 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.641689    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientPID"
Oct 30 05:30:55.643151 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.643137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.643193 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.643166    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 05:30:55.644047 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.644021    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 05:30:55.644121 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.644052    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 05:30:55.644121 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.644077    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientPID"
Oct 30 05:30:55.670858 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.670833    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-173.ec2.internal\" not found" node="ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.674318 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.674301    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-173.ec2.internal\" not found" node="ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.719387 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.719361    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6f17565975b3f18442fbd67650b5b56f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" (UID: \"6f17565975b3f18442fbd67650b5b56f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.719387 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.719388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f17565975b3f18442fbd67650b5b56f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" (UID: \"6f17565975b3f18442fbd67650b5b56f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.719552 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.719406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee22bb4b0b2b5043024e8d4645898710-config\") pod \"kube-apiserver-proxy-ip-10-0-136-173.ec2.internal\" (UID: \"ee22bb4b0b2b5043024e8d4645898710\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.741742 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.741724    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:55.819900 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.819874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6f17565975b3f18442fbd67650b5b56f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" (UID: \"6f17565975b3f18442fbd67650b5b56f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.819900 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.819903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f17565975b3f18442fbd67650b5b56f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" (UID: \"6f17565975b3f18442fbd67650b5b56f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.820062 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.819920    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee22bb4b0b2b5043024e8d4645898710-config\") pod \"kube-apiserver-proxy-ip-10-0-136-173.ec2.internal\" (UID: \"ee22bb4b0b2b5043024e8d4645898710\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.820062 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.819983    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6f17565975b3f18442fbd67650b5b56f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" (UID: \"6f17565975b3f18442fbd67650b5b56f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.820062 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.819990    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee22bb4b0b2b5043024e8d4645898710-config\") pod \"kube-apiserver-proxy-ip-10-0-136-173.ec2.internal\" (UID: \"ee22bb4b0b2b5043024e8d4645898710\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.820062 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.819990    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f17565975b3f18442fbd67650b5b56f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" (UID: \"6f17565975b3f18442fbd67650b5b56f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.842119 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.842061    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:55.942365 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:55.942335    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:55.974527 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.974501    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:55.977538 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:55.977521    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:56.042598 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:56.042565    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:56.143179 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:56.143125    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:56.243662 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:56.243637    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:56.344384 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:56.344354    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:56.354474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.354450    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 05:30:56.354647 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.354606    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 05:30:56.445052 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:56.444992    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:56.510772 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.510719    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 05:25:55 +0000 UTC" deadline="2028-04-24 13:37:16.666160472 +0000 UTC"
Oct 30 05:30:56.510772 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.510769    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13016h6m20.155396635s"
Oct 30 05:30:56.520970 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.520949    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 05:30:56.522350 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.522329    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 05:30:56.522444 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.522366    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 05:30:56.522444 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.522378    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientPID"
Oct 30 05:30:56.524646 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:56.524606    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee22bb4b0b2b5043024e8d4645898710.slice/crio-ee0e9dd0e3fbb8cec8916714da03b17114aa8711de5614358f41153935f4eeef WatchSource:0}: Error finding container ee0e9dd0e3fbb8cec8916714da03b17114aa8711de5614358f41153935f4eeef: Status 404 returned error can't find the container with id ee0e9dd0e3fbb8cec8916714da03b17114aa8711de5614358f41153935f4eeef
Oct 30 05:30:56.525220 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:56.525198    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f17565975b3f18442fbd67650b5b56f.slice/crio-81b6c9eea343a12127abee87744a659b4bbfc749aa6ac9f7ced7926d54a222bb WatchSource:0}: Error finding container 81b6c9eea343a12127abee87744a659b4bbfc749aa6ac9f7ced7926d54a222bb: Status 404 returned error can't find the container with id 81b6c9eea343a12127abee87744a659b4bbfc749aa6ac9f7ced7926d54a222bb
Oct 30 05:30:56.529793 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.529780    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 05:30:56.541651 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.541613    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal" event={"ID":"ee22bb4b0b2b5043024e8d4645898710","Type":"ContainerStarted","Data":"ee0e9dd0e3fbb8cec8916714da03b17114aa8711de5614358f41153935f4eeef"}
Oct 30 05:30:56.543744 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.543714    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal" event={"ID":"6f17565975b3f18442fbd67650b5b56f","Type":"ContainerStarted","Data":"81b6c9eea343a12127abee87744a659b4bbfc749aa6ac9f7ced7926d54a222bb"}
Oct 30 05:30:56.545113 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:56.545097    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:56.645188 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:56.645151    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:56.745723 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:56.745640    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:56.846353 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:56.846324    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:30:56.850492 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.850464    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 05:30:56.850492 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.850468    2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 05:30:56.917799 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.917754    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:56.942942 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.942912    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 05:30:56.944519 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.944492    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:30:56.964764 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:56.964739    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 05:30:57.036829 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.036801    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 05:30:57.391944 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.391874    2580 apiserver.go:52] "Watching apiserver"
Oct 30 05:30:57.417689 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.417656    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 05:30:57.421963 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.421938    2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 05:30:57.422334 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.422313    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-mtj5z","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal","openshift-multus/multus-additional-cni-plugins-g6l5g","kube-system/global-pull-secret-syncer-kkcc6","kube-system/konnectivity-agent-bfpdf","openshift-cluster-node-tuning-operator/tuned-9gcnl","openshift-multus/network-metrics-daemon-pfdj6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897","openshift-multus/multus-cbf62","openshift-network-diagnostics/network-check-target-fdfg7","openshift-ovn-kubernetes/ovnkube-node-hpgtg","kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal","openshift-image-registry/node-ca-qw6mh","openshift-network-operator/iptables-alerter-fvfm6"]
Oct 30 05:30:57.424996 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.424972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:30:57.425101 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:57.425067    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:30:57.427368 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.427346    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:30:57.429799 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.429779    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.432261 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.432239    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:30:57.432365 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:57.432318    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:30:57.434924 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.434903    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:30:57.437417 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.437396    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.439562 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.439544    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.442804 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.442779    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:30:57.444245 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.444226    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:30:57.446326 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.446040    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 05:30:57.446326 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.446153    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 05:30:57.446326 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.446197    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 05:30:57.446326 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.446290    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f8j26\""
Oct 30 05:30:57.447581 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.446948    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jtd4z\""
Oct 30 05:30:57.447581 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.447149    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 05:30:57.447581 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.447207    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 05:30:57.447581 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.447254    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 05:30:57.447581 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.447299    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 05:30:57.447581 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.447341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:30:57.447581 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.447354    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 05:30:57.447581 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:57.447390    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:30:57.450430 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.449567    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.451110 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.450844    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 05:30:57.451110 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.451034    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-2ztbt\""
Oct 30 05:30:57.451261 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.451139    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 05:30:57.452050 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.452031    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.452569 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.452539    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:30:57.453113 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.453002    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 05:30:57.453113 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.453087    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-k22sc\""
Oct 30 05:30:57.454678 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.454620    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 05:30:57.455439 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.455414    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-vg6rp\""
Oct 30 05:30:57.458774 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.458722    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 05:30:57.458982 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.458956    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xpx6p\""
Oct 30 05:30:57.458982 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.458975    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 05:30:57.460418 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.460393    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 05:30:57.460418 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.460400    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gf7fl\""
Oct 30 05:30:57.460564 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.460483    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 05:30:57.460564 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.460505    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:30:57.469176 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.467455    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 05:30:57.469176 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.467528    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wqng9"
Oct 30 05:30:57.478074 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.478050    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wqng9"
Oct 30 05:30:57.519278 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.519254    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 05:30:57.529135 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529104    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.529276 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4c2b487-ec63-4b07-a11c-781cb9afd37d-hosts-file\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:30:57.529276 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-host\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:30:57.529276 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529187    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7319282a-25bd-402e-a55c-8cc81da483c1-konnectivity-ca\") pod \"konnectivity-agent-bfpdf\" (UID: \"7319282a-25bd-402e-a55c-8cc81da483c1\") " pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:30:57.529276 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-conf-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.529276 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-multus-certs\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.529276 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-run\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-sys\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529302    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96hw6\" (UniqueName: \"kubernetes.io/projected/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-kube-api-access-96hw6\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-k8s-cni-cncf-io\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-systemd-units\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529361    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-var-lib-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-node-log\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529433    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-system-cni-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-socket-dir-parent\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-kubelet\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529512    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-etc-kubernetes\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-run-netns\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02a30d07-33cc-4be0-a920-8e839f8a493b-ovn-node-metrics-cert\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.529594 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-ovnkube-script-lib\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529611    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-device-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-lib-modules\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-netns\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-daemon-config\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rznfw\" (UniqueName: \"kubernetes.io/projected/02a30d07-33cc-4be0-a920-8e839f8a493b-kube-api-access-rznfw\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-kubelet-config\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529873    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-hostroot\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.529956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-var-lib-kubelet\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c2414f4-fa94-4054-9b41-b264e6088e67-cni-binary-copy\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-slash\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44rjf\" (UniqueName: \"kubernetes.io/projected/60bf362c-030f-4adf-9878-842b2fdc326b-kube-api-access-44rjf\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-socket-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.530281 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-registration-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7319282a-25bd-402e-a55c-8cc81da483c1-agent-certs\") pod \"konnectivity-agent-bfpdf\" (UID: \"7319282a-25bd-402e-a55c-8cc81da483c1\") " pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4c2b487-ec63-4b07-a11c-781cb9afd37d-tmp-dir\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7cjr\" (UniqueName: \"kubernetes.io/projected/0421f010-8e05-4896-b171-301e85964bb5-kube-api-access-s7cjr\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530320    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-modprobe-d\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysconfig\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-systemd\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-tmp\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530419    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrgd8\" (UniqueName: \"kubernetes.io/projected/1c2414f4-fa94-4054-9b41-b264e6088e67-kube-api-access-mrgd8\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-cnibin\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-dbus\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-cni-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-cni-multus\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530563    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysctl-conf\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.531056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-ovnkube-config\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530737    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-env-overrides\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77k6x\" (UniqueName: \"kubernetes.io/projected/b4c2b487-ec63-4b07-a11c-781cb9afd37d-kube-api-access-77k6x\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-kubernetes\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-cnibin\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-cni-bin\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-cni-binary-copy\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.530980    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6b6c8b1-dea3-45a1-b78f-676de9310b70-iptables-alerter-script\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6b6c8b1-dea3-45a1-b78f-676de9310b70-host-slash\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zcgx\" (UniqueName: \"kubernetes.io/projected/a6b6c8b1-dea3-45a1-b78f-676de9310b70-kube-api-access-8zcgx\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wkpm\" (UniqueName: \"kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm\") pod \"network-check-target-fdfg7\" (UID: \"f82afdb2-9cea-479b-8d0b-88a814886c3a\") " pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-serviceca\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djp28\" (UniqueName: \"kubernetes.io/projected/e71a9464-fdd4-491d-a0ce-51519a35486d-kube-api-access-djp28\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysctl-d\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.531872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-tuned\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.532674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531476    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-systemd\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.532674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-etc-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.532674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-ovn\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.532674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-log-socket\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.532674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-cni-bin\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.532674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531692    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-sys-fs\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.532674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rd8b\" (UniqueName: \"kubernetes.io/projected/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-kube-api-access-8rd8b\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:30:57.532674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-host\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.532674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-kubelet\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.532674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-run-ovn-kubernetes\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.532674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-cni-netd\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.532674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.531854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-system-cni-dir\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.559324 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.559293    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 05:30:57.559634 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.559615    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 05:30:57.559747 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.559602    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 05:30:57.559747 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.559723    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 05:30:57.559863 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.559769    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 05:30:57.560046 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.560015    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 05:30:57.560162 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.560047    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mjhr7\""
Oct 30 05:30:57.560162 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.560092    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 05:30:57.560162 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.560113    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 05:30:57.560355 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.560303    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 05:30:57.560418 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.560402    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7pqn7\""
Oct 30 05:30:57.632506 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.632459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77k6x\" (UniqueName: \"kubernetes.io/projected/b4c2b487-ec63-4b07-a11c-781cb9afd37d-kube-api-access-77k6x\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:30:57.632506 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.632503    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-kubernetes\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.632782 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.632531    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-cnibin\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.632782 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.632590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-cni-bin\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.632782 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.632638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-cni-binary-copy\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.632782 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.632667    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.632782 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.632693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:30:57.632782 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.632719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6b6c8b1-dea3-45a1-b78f-676de9310b70-iptables-alerter-script\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633160    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-cni-bin\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.632743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6b6c8b1-dea3-45a1-b78f-676de9310b70-host-slash\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633351    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zcgx\" (UniqueName: \"kubernetes.io/projected/a6b6c8b1-dea3-45a1-b78f-676de9310b70-kube-api-access-8zcgx\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wkpm\" (UniqueName: \"kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm\") pod \"network-check-target-fdfg7\" (UID: \"f82afdb2-9cea-479b-8d0b-88a814886c3a\") " pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-serviceca\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djp28\" (UniqueName: \"kubernetes.io/projected/e71a9464-fdd4-491d-a0ce-51519a35486d-kube-api-access-djp28\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysctl-d\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-tuned\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-systemd\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-etc-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-ovn\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633537    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-kubernetes\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-log-socket\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-cni-bin\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-sys-fs\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633625    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-cnibin\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.636544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rd8b\" (UniqueName: \"kubernetes.io/projected/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-kube-api-access-8rd8b\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633688    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-host\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-kubelet\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-run-ovn-kubernetes\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-cni-netd\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633788    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-system-cni-dir\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633816    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4c2b487-ec63-4b07-a11c-781cb9afd37d-hosts-file\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-host\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7319282a-25bd-402e-a55c-8cc81da483c1-konnectivity-ca\") pod \"konnectivity-agent-bfpdf\" (UID: \"7319282a-25bd-402e-a55c-8cc81da483c1\") " pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-conf-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-multus-certs\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-run\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633949    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-sys\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96hw6\" (UniqueName: \"kubernetes.io/projected/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-kube-api-access-96hw6\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.633999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-k8s-cni-cncf-io\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-systemd-units\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.637460 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-var-lib-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-node-log\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-system-cni-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-socket-dir-parent\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-kubelet\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-etc-kubernetes\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-run-netns\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634321    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02a30d07-33cc-4be0-a920-8e839f8a493b-ovn-node-metrics-cert\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-ovnkube-script-lib\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-device-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634386    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-lib-modules\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634410    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-netns\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-daemon-config\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rznfw\" (UniqueName: \"kubernetes.io/projected/02a30d07-33cc-4be0-a920-8e839f8a493b-kube-api-access-rznfw\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-kubelet-config\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634523    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6b6c8b1-dea3-45a1-b78f-676de9310b70-host-slash\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:30:57.638299 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634547    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-kubelet-config\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-hostroot\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:57.634628    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634690    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysctl-d\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-var-lib-kubelet\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:57.634707    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret podName:7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa nodeName:}" failed. No retries permitted until 2026-10-30 05:30:58.134683597 +0000 UTC m=+3.073057981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret") pod "global-pull-secret-syncer-kkcc6" (UID: "7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa") : object "kube-system"/"original-pull-secret" not registered
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634763    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-sys\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634838    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.634870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c2414f4-fa94-4054-9b41-b264e6088e67-cni-binary-copy\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635436    2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635541    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635557    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-lib-modules\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635614    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-run-ovn-kubernetes\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635633    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-kubelet\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635688    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-conf-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635691    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-systemd\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.639203 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635728    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-etc-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635730    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-cni-netd\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635759    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-k8s-cni-cncf-io\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635771    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-cni-binary-copy\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635794    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-systemd-units\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-slash\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635812    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-slash\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635831    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-var-lib-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635846    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-ovn\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635843    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635867    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-node-log\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635897    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-log-socket\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635925    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:57.635985    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:57.636050    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs podName:60bf362c-030f-4adf-9878-842b2fdc326b nodeName:}" failed. No retries permitted until 2026-10-30 05:30:58.136025684 +0000 UTC m=+3.074400055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs") pod "network-metrics-daemon-pfdj6" (UID: "60bf362c-030f-4adf-9878-842b2fdc326b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636085    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-socket-dir-parent\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44rjf\" (UniqueName: \"kubernetes.io/projected/60bf362c-030f-4adf-9878-842b2fdc326b-kube-api-access-44rjf\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:30:57.640058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636126    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-socket-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636129    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-serviceca\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-registration-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636182    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6b6c8b1-dea3-45a1-b78f-676de9310b70-iptables-alerter-script\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7319282a-25bd-402e-a55c-8cc81da483c1-agent-certs\") pod \"konnectivity-agent-bfpdf\" (UID: \"7319282a-25bd-402e-a55c-8cc81da483c1\") " pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4c2b487-ec63-4b07-a11c-781cb9afd37d-tmp-dir\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636283    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.635984    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-system-cni-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7cjr\" (UniqueName: \"kubernetes.io/projected/0421f010-8e05-4896-b171-301e85964bb5-kube-api-access-s7cjr\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636331    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-cni-bin\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636335    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-modprobe-d\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636370    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-run\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636378    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636479    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-run-netns\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636508    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4c2b487-ec63-4b07-a11c-781cb9afd37d-hosts-file\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636536    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-registration-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.640757 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysconfig\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636571    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-systemd\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636667    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-tmp\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636721    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-netns\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636820    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-var-lib-kubelet\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrgd8\" (UniqueName: \"kubernetes.io/projected/1c2414f4-fa94-4054-9b41-b264e6088e67-kube-api-access-mrgd8\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-cnibin\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-dbus\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-cni-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-cni-multus\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636572    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-host\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysctl-conf\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637076    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c2414f4-fa94-4054-9b41-b264e6088e67-cni-binary-copy\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-ovnkube-config\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-env-overrides\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637140    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-multus-certs\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637268    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-daemon-config\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.642297 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637332    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-sys-fs\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637395    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4c2b487-ec63-4b07-a11c-781cb9afd37d-tmp-dir\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.636482    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-modprobe-d\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637450    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-etc-kubernetes\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637582    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-cni-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637651    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-cnibin\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637733    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-dbus\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637749    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysctl-conf\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637786    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-hostroot\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637794    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.637822    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-cni-multus\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.638009    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-system-cni-dir\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.638011    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-socket-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.638077    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysconfig\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.638128    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-host\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.638182    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-systemd\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.638509    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-kubelet\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.638521    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7319282a-25bd-402e-a55c-8cc81da483c1-konnectivity-ca\") pod \"konnectivity-agent-bfpdf\" (UID: \"7319282a-25bd-402e-a55c-8cc81da483c1\") " pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:30:57.643160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.638696    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-device-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.644103 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.638798    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-ovnkube-script-lib\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.644103 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.639140    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-env-overrides\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.644103 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.639575    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.644103 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.643857    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-tuned\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.644103 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.644084    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-tmp\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.647168 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.645002    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02a30d07-33cc-4be0-a920-8e839f8a493b-ovn-node-metrics-cert\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.647168 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.645101    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7319282a-25bd-402e-a55c-8cc81da483c1-agent-certs\") pod \"konnectivity-agent-bfpdf\" (UID: \"7319282a-25bd-402e-a55c-8cc81da483c1\") " pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:30:57.647168 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.645101    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-ovnkube-config\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.657719 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:57.657694    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 05:30:57.657719 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:57.657718    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 05:30:57.657877 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:57.657729    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5wkpm for pod openshift-network-diagnostics/network-check-target-fdfg7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 05:30:57.657937 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:57.657914    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm podName:f82afdb2-9cea-479b-8d0b-88a814886c3a nodeName:}" failed. No retries permitted until 2026-10-30 05:30:58.157875837 +0000 UTC m=+3.096250205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5wkpm" (UniqueName: "kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm") pod "network-check-target-fdfg7" (UID: "f82afdb2-9cea-479b-8d0b-88a814886c3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 05:30:57.659147 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.659120    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djp28\" (UniqueName: \"kubernetes.io/projected/e71a9464-fdd4-491d-a0ce-51519a35486d-kube-api-access-djp28\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.659302 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.659243    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77k6x\" (UniqueName: \"kubernetes.io/projected/b4c2b487-ec63-4b07-a11c-781cb9afd37d-kube-api-access-77k6x\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:30:57.661107 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.661064    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96hw6\" (UniqueName: \"kubernetes.io/projected/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-kube-api-access-96hw6\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.661224 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.661162    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zcgx\" (UniqueName: \"kubernetes.io/projected/a6b6c8b1-dea3-45a1-b78f-676de9310b70-kube-api-access-8zcgx\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:30:57.661911 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.661876    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44rjf\" (UniqueName: \"kubernetes.io/projected/60bf362c-030f-4adf-9878-842b2fdc326b-kube-api-access-44rjf\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:30:57.663946 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.663925    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rd8b\" (UniqueName: \"kubernetes.io/projected/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-kube-api-access-8rd8b\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:30:57.664107 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.664084    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7cjr\" (UniqueName: \"kubernetes.io/projected/0421f010-8e05-4896-b171-301e85964bb5-kube-api-access-s7cjr\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.664187 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.664091    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rznfw\" (UniqueName: \"kubernetes.io/projected/02a30d07-33cc-4be0-a920-8e839f8a493b-kube-api-access-rznfw\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.665671 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.665577    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrgd8\" (UniqueName: \"kubernetes.io/projected/1c2414f4-fa94-4054-9b41-b264e6088e67-kube-api-access-mrgd8\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.691645 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.691613    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 05:30:57.744511 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.744248    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:30:57.757341 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:57.757313    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14a30f61_33a2_4f29_ba24_fc02c0c6cc15.slice/crio-c5f940a78d2aef49f815a2dd9f61a66a2c9124d79514816b4140449224db99d9 WatchSource:0}: Error finding container c5f940a78d2aef49f815a2dd9f61a66a2c9124d79514816b4140449224db99d9: Status 404 returned error can't find the container with id c5f940a78d2aef49f815a2dd9f61a66a2c9124d79514816b4140449224db99d9
Oct 30 05:30:57.771412 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.771383    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:30:57.775803 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.775778    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:30:57.776051 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.776035    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:30:57.782255 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.782222    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:30:57.784037 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:57.784008    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode71a9464_fdd4_491d_a0ce_51519a35486d.slice/crio-234f6ba211b434eafd446ebb3a0e1e812e1f7be58cab99d97cad92fefc541a2f WatchSource:0}: Error finding container 234f6ba211b434eafd446ebb3a0e1e812e1f7be58cab99d97cad92fefc541a2f: Status 404 returned error can't find the container with id 234f6ba211b434eafd446ebb3a0e1e812e1f7be58cab99d97cad92fefc541a2f
Oct 30 05:30:57.791020 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.791001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cbf62"
Oct 30 05:30:57.791848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:57.791798    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4c2b487_ec63_4b07_a11c_781cb9afd37d.slice/crio-6568dac5d31741f81717f7f963ea0942fc4319cfeaa204feeeb730ef5fb61d83 WatchSource:0}: Error finding container 6568dac5d31741f81717f7f963ea0942fc4319cfeaa204feeeb730ef5fb61d83: Status 404 returned error can't find the container with id 6568dac5d31741f81717f7f963ea0942fc4319cfeaa204feeeb730ef5fb61d83
Oct 30 05:30:57.794706 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:57.794680    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7319282a_25bd_402e_a55c_8cc81da483c1.slice/crio-6255a142722f21e56116020f8e3ac7cbb6f18f187ab0379bc79e9e62907c9e54 WatchSource:0}: Error finding container 6255a142722f21e56116020f8e3ac7cbb6f18f187ab0379bc79e9e62907c9e54: Status 404 returned error can't find the container with id 6255a142722f21e56116020f8e3ac7cbb6f18f187ab0379bc79e9e62907c9e54
Oct 30 05:30:57.798656 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.798601    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:30:57.800296 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:57.800260    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd432e7e1_19d3_4f6d_8d47_51f8d6a3d166.slice/crio-111847702b4a2e1675b3ef14ebc814183b97b50e48c465c7316ce234e02091d7 WatchSource:0}: Error finding container 111847702b4a2e1675b3ef14ebc814183b97b50e48c465c7316ce234e02091d7: Status 404 returned error can't find the container with id 111847702b4a2e1675b3ef14ebc814183b97b50e48c465c7316ce234e02091d7
Oct 30 05:30:57.803735 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:57.803402    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c2414f4_fa94_4054_9b41_b264e6088e67.slice/crio-406513e1463071b6445938af507dd56bca6de79786a14a04ef7ddfc876858b95 WatchSource:0}: Error finding container 406513e1463071b6445938af507dd56bca6de79786a14a04ef7ddfc876858b95: Status 404 returned error can't find the container with id 406513e1463071b6445938af507dd56bca6de79786a14a04ef7ddfc876858b95
Oct 30 05:30:57.811074 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:57.811023    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6b6c8b1_dea3_45a1_b78f_676de9310b70.slice/crio-0c11cc7919f86587553eabfc7f1e38ecf45ca69f6da0796ec26d59dd919f3a07 WatchSource:0}: Error finding container 0c11cc7919f86587553eabfc7f1e38ecf45ca69f6da0796ec26d59dd919f3a07: Status 404 returned error can't find the container with id 0c11cc7919f86587553eabfc7f1e38ecf45ca69f6da0796ec26d59dd919f3a07
Oct 30 05:30:57.826728 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.826459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:30:57.831655 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:57.831462    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:30:57.840807 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:57.840779    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0421f010_8e05_4896_b171_301e85964bb5.slice/crio-fb581939b0d830c6cf834034c0eae53edc3496352ef73bb25cfa79cf541cd4fd WatchSource:0}: Error finding container fb581939b0d830c6cf834034c0eae53edc3496352ef73bb25cfa79cf541cd4fd: Status 404 returned error can't find the container with id fb581939b0d830c6cf834034c0eae53edc3496352ef73bb25cfa79cf541cd4fd
Oct 30 05:30:57.841589 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:30:57.841563    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a30d07_33cc_4be0_a920_8e839f8a493b.slice/crio-af946f0caace3ff447d5c528d8fa32b2c6f7164c1d924a788f9fd15115106166 WatchSource:0}: Error finding container af946f0caace3ff447d5c528d8fa32b2c6f7164c1d924a788f9fd15115106166: Status 404 returned error can't find the container with id af946f0caace3ff447d5c528d8fa32b2c6f7164c1d924a788f9fd15115106166
Oct 30 05:30:58.144576 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.144165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:30:58.144576 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.144541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:30:58.144576 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:58.144318    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 05:30:58.144860 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:58.144650    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs podName:60bf362c-030f-4adf-9878-842b2fdc326b nodeName:}" failed. No retries permitted until 2026-10-30 05:30:59.144627098 +0000 UTC m=+4.083001460 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs") pod "network-metrics-daemon-pfdj6" (UID: "60bf362c-030f-4adf-9878-842b2fdc326b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 05:30:58.144860 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:58.144696    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 05:30:58.144860 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:58.144731    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret podName:7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa nodeName:}" failed. No retries permitted until 2026-10-30 05:30:59.144722886 +0000 UTC m=+4.083097276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret") pod "global-pull-secret-syncer-kkcc6" (UID: "7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa") : object "kube-system"/"original-pull-secret" not registered
Oct 30 05:30:58.245657 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.244962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wkpm\" (UniqueName: \"kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm\") pod \"network-check-target-fdfg7\" (UID: \"f82afdb2-9cea-479b-8d0b-88a814886c3a\") " pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:30:58.245657 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:58.245180    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 05:30:58.245657 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:58.245204    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 05:30:58.245657 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:58.245219    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5wkpm for pod openshift-network-diagnostics/network-check-target-fdfg7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 05:30:58.245657 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:58.245278    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm podName:f82afdb2-9cea-479b-8d0b-88a814886c3a nodeName:}" failed. No retries permitted until 2026-10-30 05:30:59.245258811 +0000 UTC m=+4.183633184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5wkpm" (UniqueName: "kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm") pod "network-check-target-fdfg7" (UID: "f82afdb2-9cea-479b-8d0b-88a814886c3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 05:30:58.481924 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.481756    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 05:25:57 +0000 UTC" deadline="2028-06-26 18:16:22.2357079 +0000 UTC"
Oct 30 05:30:58.481924 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.481827    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14532h45m23.753885227s"
Oct 30 05:30:58.538506 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.537931    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:30:58.538506 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:58.538130    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:30:58.572790 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.572752    2580 generic.go:356] "Generic (PLEG): container finished" podID="6f17565975b3f18442fbd67650b5b56f" containerID="b5cbaa1666f97cc808001b56e15afbb07cf03552c94130d44e6d4c321f85f74d" exitCode=0
Oct 30 05:30:58.572934 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.572903    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal" event={"ID":"6f17565975b3f18442fbd67650b5b56f","Type":"ContainerDied","Data":"b5cbaa1666f97cc808001b56e15afbb07cf03552c94130d44e6d4c321f85f74d"}
Oct 30 05:30:58.581263 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.581224    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal" event={"ID":"ee22bb4b0b2b5043024e8d4645898710","Type":"ContainerStarted","Data":"e508dc326c3dd40a245acbd04d22c1afc57adec21e74f0fa0127b638b45e6802"}
Oct 30 05:30:58.606712 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.605250    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897" event={"ID":"0421f010-8e05-4896-b171-301e85964bb5","Type":"ContainerStarted","Data":"fb581939b0d830c6cf834034c0eae53edc3496352ef73bb25cfa79cf541cd4fd"}
Oct 30 05:30:58.635529 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.635397    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fvfm6" event={"ID":"a6b6c8b1-dea3-45a1-b78f-676de9310b70","Type":"ContainerStarted","Data":"0c11cc7919f86587553eabfc7f1e38ecf45ca69f6da0796ec26d59dd919f3a07"}
Oct 30 05:30:58.648236 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.648203    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cbf62" event={"ID":"1c2414f4-fa94-4054-9b41-b264e6088e67","Type":"ContainerStarted","Data":"406513e1463071b6445938af507dd56bca6de79786a14a04ef7ddfc876858b95"}
Oct 30 05:30:58.668054 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.668011    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bfpdf" event={"ID":"7319282a-25bd-402e-a55c-8cc81da483c1","Type":"ContainerStarted","Data":"6255a142722f21e56116020f8e3ac7cbb6f18f187ab0379bc79e9e62907c9e54"}
Oct 30 05:30:58.672174 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.671639    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal" podStartSLOduration=1.671619307 podStartE2EDuration="1.671619307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 05:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:30:58.670660743 +0000 UTC m=+3.609035143" watchObservedRunningTime="2026-10-30 05:30:58.671619307 +0000 UTC m=+3.609993699"
Oct 30 05:30:58.681560 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.681461    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6l5g" event={"ID":"e71a9464-fdd4-491d-a0ce-51519a35486d","Type":"ContainerStarted","Data":"234f6ba211b434eafd446ebb3a0e1e812e1f7be58cab99d97cad92fefc541a2f"}
Oct 30 05:30:58.687908 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.687871    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qw6mh" event={"ID":"14a30f61-33a2-4f29-ba24-fc02c0c6cc15","Type":"ContainerStarted","Data":"c5f940a78d2aef49f815a2dd9f61a66a2c9124d79514816b4140449224db99d9"}
Oct 30 05:30:58.699082 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.699044    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg" event={"ID":"02a30d07-33cc-4be0-a920-8e839f8a493b","Type":"ContainerStarted","Data":"af946f0caace3ff447d5c528d8fa32b2c6f7164c1d924a788f9fd15115106166"}
Oct 30 05:30:58.716872 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.716551    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9gcnl" event={"ID":"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166","Type":"ContainerStarted","Data":"111847702b4a2e1675b3ef14ebc814183b97b50e48c465c7316ce234e02091d7"}
Oct 30 05:30:58.720538 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:58.720461    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mtj5z" event={"ID":"b4c2b487-ec63-4b07-a11c-781cb9afd37d","Type":"ContainerStarted","Data":"6568dac5d31741f81717f7f963ea0942fc4319cfeaa204feeeb730ef5fb61d83"}
Oct 30 05:30:59.153998 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:59.153952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:30:59.154170 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:59.154040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:30:59.154170 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:59.154106    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 05:30:59.154170 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:59.154157    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 05:30:59.154337 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:59.154186    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret podName:7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa nodeName:}" failed. No retries permitted until 2026-10-30 05:31:01.154171561 +0000 UTC m=+6.092545923 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret") pod "global-pull-secret-syncer-kkcc6" (UID: "7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa") : object "kube-system"/"original-pull-secret" not registered
Oct 30 05:30:59.154337 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:59.154209    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs podName:60bf362c-030f-4adf-9878-842b2fdc326b nodeName:}" failed. No retries permitted until 2026-10-30 05:31:01.154192787 +0000 UTC m=+6.092567161 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs") pod "network-metrics-daemon-pfdj6" (UID: "60bf362c-030f-4adf-9878-842b2fdc326b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 05:30:59.254934 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:59.254899    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wkpm\" (UniqueName: \"kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm\") pod \"network-check-target-fdfg7\" (UID: \"f82afdb2-9cea-479b-8d0b-88a814886c3a\") " pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:30:59.255161 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:59.255050    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 05:30:59.255161 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:59.255067    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 05:30:59.255161 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:59.255079    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5wkpm for pod openshift-network-diagnostics/network-check-target-fdfg7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 05:30:59.255161 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:59.255134    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm podName:f82afdb2-9cea-479b-8d0b-88a814886c3a nodeName:}" failed. No retries permitted until 2026-10-30 05:31:01.255115517 +0000 UTC m=+6.193489922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5wkpm" (UniqueName: "kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm") pod "network-check-target-fdfg7" (UID: "f82afdb2-9cea-479b-8d0b-88a814886c3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 05:30:59.482650 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:59.482531    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 05:25:57 +0000 UTC" deadline="2028-05-04 23:18:26.943823156 +0000 UTC"
Oct 30 05:30:59.482650 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:59.482571    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13265h47m27.461257582s"
Oct 30 05:30:59.540739 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:59.540704    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:30:59.540897 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:59.540857    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:30:59.541258 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:59.541239    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:30:59.541354 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:30:59.541336    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:30:59.731526 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:59.731459    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal" event={"ID":"6f17565975b3f18442fbd67650b5b56f","Type":"ContainerStarted","Data":"8cab33c9540f3a3ee6bf3594699fd4402bf5cab06448e0e6fa441d0beb2b6463"}
Oct 30 05:30:59.774029 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:30:59.773912    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal" podStartSLOduration=2.773892942 podStartE2EDuration="2.773892942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 05:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:30:59.772819526 +0000 UTC m=+4.711193910" watchObservedRunningTime="2026-10-30 05:30:59.773892942 +0000 UTC m=+4.712267328"
Oct 30 05:31:00.537773 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:00.537716    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:00.538236 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:00.537871    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:00.751002 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:00.750866    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6l5g" event={"ID":"e71a9464-fdd4-491d-a0ce-51519a35486d","Type":"ContainerStarted","Data":"44b23df4824142e19d11d9220314bf4a4b039586b9488ea42c1bbd7dbe4ced0e"}
Oct 30 05:31:01.171350 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:01.170211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:01.171350 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:01.170313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:01.171350 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:01.170445    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 05:31:01.171350 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:01.170508    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs podName:60bf362c-030f-4adf-9878-842b2fdc326b nodeName:}" failed. No retries permitted until 2026-10-30 05:31:05.170490407 +0000 UTC m=+10.108864779 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs") pod "network-metrics-daemon-pfdj6" (UID: "60bf362c-030f-4adf-9878-842b2fdc326b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 05:31:01.171350 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:01.170937    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 05:31:01.171350 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:01.170991    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret podName:7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa nodeName:}" failed. No retries permitted until 2026-10-30 05:31:05.170976435 +0000 UTC m=+10.109350801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret") pod "global-pull-secret-syncer-kkcc6" (UID: "7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa") : object "kube-system"/"original-pull-secret" not registered
Oct 30 05:31:01.271629 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:01.271581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wkpm\" (UniqueName: \"kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm\") pod \"network-check-target-fdfg7\" (UID: \"f82afdb2-9cea-479b-8d0b-88a814886c3a\") " pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:01.271812 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:01.271787    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 05:31:01.271812 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:01.271807    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 05:31:01.271918 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:01.271819    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5wkpm for pod openshift-network-diagnostics/network-check-target-fdfg7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 05:31:01.271918 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:01.271879    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm podName:f82afdb2-9cea-479b-8d0b-88a814886c3a nodeName:}" failed. No retries permitted until 2026-10-30 05:31:05.271859591 +0000 UTC m=+10.210233967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5wkpm" (UniqueName: "kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm") pod "network-check-target-fdfg7" (UID: "f82afdb2-9cea-479b-8d0b-88a814886c3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 05:31:01.540467 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:01.540387    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:01.540960 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:01.540523    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:31:01.541039 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:01.540974    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:01.541152 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:01.541117    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:31:01.755899 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:01.755788    2580 generic.go:356] "Generic (PLEG): container finished" podID="e71a9464-fdd4-491d-a0ce-51519a35486d" containerID="44b23df4824142e19d11d9220314bf4a4b039586b9488ea42c1bbd7dbe4ced0e" exitCode=0
Oct 30 05:31:01.755899 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:01.755842    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6l5g" event={"ID":"e71a9464-fdd4-491d-a0ce-51519a35486d","Type":"ContainerDied","Data":"44b23df4824142e19d11d9220314bf4a4b039586b9488ea42c1bbd7dbe4ced0e"}
Oct 30 05:31:02.538185 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:02.537697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:02.538185 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:02.537848    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:02.765815 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:02.765041    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bfpdf" event={"ID":"7319282a-25bd-402e-a55c-8cc81da483c1","Type":"ContainerStarted","Data":"c06ae191f3b523bdbec7b88d4db8b6f7a5fd2887c6cd74e2466e8a7992b7dcd5"}
Oct 30 05:31:02.773488 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:02.773030    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qw6mh" event={"ID":"14a30f61-33a2-4f29-ba24-fc02c0c6cc15","Type":"ContainerStarted","Data":"ec3def7c703f3df9a176ca0560daf0f5f7eefbc93c3bb08f7c1debf1c810edb6"}
Oct 30 05:31:02.845003 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:02.844913    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bfpdf" podStartSLOduration=2.978491989 podStartE2EDuration="6.84489468s" totalImagesPullingTime="3.866402691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:02.806739064 +0000 UTC m=+7.745113443" watchObservedRunningTime="2026-10-30 05:31:02.84489468 +0000 UTC m=+7.783269056"
Oct 30 05:31:03.539325 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:03.539295    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:03.539480 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:03.539342    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:03.539480 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:03.539448    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:31:03.539650 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:03.539581    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:31:04.537780 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:04.537698    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:04.538276 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:04.537831    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:04.786969 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:04.786913    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897" event={"ID":"0421f010-8e05-4896-b171-301e85964bb5","Type":"ContainerStarted","Data":"1b2c3fcf4edb1584c9f7eb0fae2c5fa155b0433d8e80d7e840ba9ce6633424c2"}
Oct 30 05:31:05.209716 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.208891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:05.209716 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.208985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:05.209716 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:05.209106    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 05:31:05.209716 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:05.209173    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs podName:60bf362c-030f-4adf-9878-842b2fdc326b nodeName:}" failed. No retries permitted until 2026-10-30 05:31:13.209154511 +0000 UTC m=+18.147528873 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs") pod "network-metrics-daemon-pfdj6" (UID: "60bf362c-030f-4adf-9878-842b2fdc326b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 05:31:05.209716 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:05.209624    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 05:31:05.209716 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:05.209675    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret podName:7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa nodeName:}" failed. No retries permitted until 2026-10-30 05:31:13.209660181 +0000 UTC m=+18.148034541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret") pod "global-pull-secret-syncer-kkcc6" (UID: "7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa") : object "kube-system"/"original-pull-secret" not registered
Oct 30 05:31:05.286061 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.286023    2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 05:31:05.310218 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.310181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wkpm\" (UniqueName: \"kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm\") pod \"network-check-target-fdfg7\" (UID: \"f82afdb2-9cea-479b-8d0b-88a814886c3a\") " pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:05.310428 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:05.310410    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 05:31:05.310488 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:05.310435    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 05:31:05.310488 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:05.310448    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5wkpm for pod openshift-network-diagnostics/network-check-target-fdfg7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 05:31:05.310619 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:05.310502    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm podName:f82afdb2-9cea-479b-8d0b-88a814886c3a nodeName:}" failed. No retries permitted until 2026-10-30 05:31:13.310485381 +0000 UTC m=+18.248859750 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5wkpm" (UniqueName: "kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm") pod "network-check-target-fdfg7" (UID: "f82afdb2-9cea-479b-8d0b-88a814886c3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 05:31:05.492381 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.492304    2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T05:31:05.286041597Z","UUID":"7c2e7888-e5ae-4afd-b950-491214b77bc6","Handler":null,"Name":"","Endpoint":""}
Oct 30 05:31:05.494325 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.494308    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 05:31:05.494410 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.494333    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 05:31:05.538850 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.538408    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:05.538850 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:05.538532    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:31:05.539603 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.539564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:05.539696 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:05.539663    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:31:05.737876 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.737841    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:31:05.739329 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.739109    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:31:05.779374 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.778675    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qw6mh" podStartSLOduration=5.068532632 podStartE2EDuration="9.778660139s" totalImagesPullingTime="4.710127507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 05:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:02.848883813 +0000 UTC m=+7.787258215" watchObservedRunningTime="2026-10-30 05:31:05.778660139 +0000 UTC m=+10.717034557"
Oct 30 05:31:05.792567 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.792526    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cbf62" event={"ID":"1c2414f4-fa94-4054-9b41-b264e6088e67","Type":"ContainerStarted","Data":"4ea7c87422b243ede6ae90614ad2e15516c7e58b11949c2e6f4dda22243e0e33"}
Oct 30 05:31:05.795713 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.795202    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mtj5z" event={"ID":"b4c2b487-ec63-4b07-a11c-781cb9afd37d","Type":"ContainerStarted","Data":"f76e280197a7c4e31f4d2ece19d5cfac3a71003e3e59a10a2da2b750454954cd"}
Oct 30 05:31:05.797952 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.797883    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897" event={"ID":"0421f010-8e05-4896-b171-301e85964bb5","Type":"ContainerStarted","Data":"06f74c1370b52c4f3932aae5fb77d536106ed44e0bf61cd5abc3d62b1af7b6bf"}
Oct 30 05:31:05.836576 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:05.835978    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-cbf62" podStartSLOduration=2.150287604 podStartE2EDuration="9.835966385s" totalImagesPullingTime="7.685678781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 05:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:05.835835199 +0000 UTC m=+10.774209579" watchObservedRunningTime="2026-10-30 05:31:05.835966385 +0000 UTC m=+10.774340764"
Oct 30 05:31:06.538717 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:06.538472    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:06.539174 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:06.538840    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:06.802896 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:06.802332    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897" event={"ID":"0421f010-8e05-4896-b171-301e85964bb5","Type":"ContainerStarted","Data":"487c4cda40c94cfa1883ac2079e53700ba45605f47194937bcc17d7742246ee9"}
Oct 30 05:31:06.804426 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:06.804395    2580 generic.go:356] "Generic (PLEG): container finished" podID="e71a9464-fdd4-491d-a0ce-51519a35486d" containerID="e681a8157306c319fad1feda6ad19b73845bd932e4132f6d7a20cd759699db22" exitCode=0
Oct 30 05:31:06.805120 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:06.804648    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6l5g" event={"ID":"e71a9464-fdd4-491d-a0ce-51519a35486d","Type":"ContainerDied","Data":"e681a8157306c319fad1feda6ad19b73845bd932e4132f6d7a20cd759699db22"}
Oct 30 05:31:06.845186 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:06.845123    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-mtj5z" podStartSLOduration=3.14531249 podStartE2EDuration="10.845107191s" totalImagesPullingTime="7.699794701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 05:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:05.874517418 +0000 UTC m=+10.812891798" watchObservedRunningTime="2026-10-30 05:31:06.845107191 +0000 UTC m=+11.783481569"
Oct 30 05:31:06.896769 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:06.896709    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897" podStartSLOduration=2.876428856 podStartE2EDuration="10.896685737s" totalImagesPullingTime="8.020256881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 05:30:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:06.845980795 +0000 UTC m=+11.784355174" watchObservedRunningTime="2026-10-30 05:31:06.896685737 +0000 UTC m=+11.835060145"
Oct 30 05:31:07.538559 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:07.538142    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:07.538559 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:07.538264    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:31:07.538769 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:07.538673    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:07.538801 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:07.538767    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:31:07.809303 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:07.809226    2580 generic.go:356] "Generic (PLEG): container finished" podID="e71a9464-fdd4-491d-a0ce-51519a35486d" containerID="5dc76552cfcf53c013fc56ed27faaded265ad672f530d660407c1cad9981ccb5" exitCode=0
Oct 30 05:31:07.809303 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:07.809280    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6l5g" event={"ID":"e71a9464-fdd4-491d-a0ce-51519a35486d","Type":"ContainerDied","Data":"5dc76552cfcf53c013fc56ed27faaded265ad672f530d660407c1cad9981ccb5"}
Oct 30 05:31:07.811058 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:07.811032    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fvfm6" event={"ID":"a6b6c8b1-dea3-45a1-b78f-676de9310b70","Type":"ContainerStarted","Data":"162f65a25996f274c55d876cd8af4c9bcf2daaf456920da54643a89d6a9fcdfd"}
Oct 30 05:31:07.918881 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:07.918818    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fvfm6" podStartSLOduration=3.909835729 podStartE2EDuration="11.918798323s" totalImagesPullingTime="8.008962594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 05:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:07.918166599 +0000 UTC m=+12.856540992" watchObservedRunningTime="2026-10-30 05:31:07.918798323 +0000 UTC m=+12.857172704"
Oct 30 05:31:08.537789 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:08.537751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:08.537940 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:08.537883    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:08.815703 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:08.815609    2580 generic.go:356] "Generic (PLEG): container finished" podID="e71a9464-fdd4-491d-a0ce-51519a35486d" containerID="25c29d007fd1a4b08bc1a56919b26d380dffcc03af69c2fa1295b0d4f9bee077" exitCode=0
Oct 30 05:31:08.816211 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:08.815902    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6l5g" event={"ID":"e71a9464-fdd4-491d-a0ce-51519a35486d","Type":"ContainerDied","Data":"25c29d007fd1a4b08bc1a56919b26d380dffcc03af69c2fa1295b0d4f9bee077"}
Oct 30 05:31:08.818267 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:08.818242    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9gcnl" event={"ID":"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166","Type":"ContainerStarted","Data":"a675a0f829d3f60de82814fd2002f2ac0c6201beefaee7d860877a38ecff10e9"}
Oct 30 05:31:08.930795 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:08.930737    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9gcnl" podStartSLOduration=2.687533459 podStartE2EDuration="12.930724623s" totalImagesPullingTime="10.243191164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 05:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:08.930428649 +0000 UTC m=+13.868803054" watchObservedRunningTime="2026-10-30 05:31:08.930724623 +0000 UTC m=+13.869099006"
Oct 30 05:31:09.537893 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:09.537773    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:09.538422 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:09.538380    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:31:09.538709 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:09.538689    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:09.538936 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:09.538915    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:31:10.537683 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:10.537648    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:10.538331 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:10.537798    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:10.826501 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:10.826263    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6l5g" event={"ID":"e71a9464-fdd4-491d-a0ce-51519a35486d","Type":"ContainerStarted","Data":"2f25d09378ce9139562b573776a154398a698ac217207a64e98d121205465b29"}
Oct 30 05:31:11.541251 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:11.541218    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:11.541251 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:11.541240    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:11.541762 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:11.541348    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:31:11.541762 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:11.541471    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:31:11.830967 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:11.830894    2580 generic.go:356] "Generic (PLEG): container finished" podID="e71a9464-fdd4-491d-a0ce-51519a35486d" containerID="2f25d09378ce9139562b573776a154398a698ac217207a64e98d121205465b29" exitCode=0
Oct 30 05:31:11.830967 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:11.830933    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6l5g" event={"ID":"e71a9464-fdd4-491d-a0ce-51519a35486d","Type":"ContainerDied","Data":"2f25d09378ce9139562b573776a154398a698ac217207a64e98d121205465b29"}
Oct 30 05:31:12.538490 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:12.538467    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:12.538599 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:12.538568    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:12.836811 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:12.836637    2580 generic.go:356] "Generic (PLEG): container finished" podID="e71a9464-fdd4-491d-a0ce-51519a35486d" containerID="cb92081d44e779c1995c55501d5f98901ddd3f07f0de92e04c6baceafaf829f9" exitCode=0
Oct 30 05:31:12.837357 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:12.836720    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6l5g" event={"ID":"e71a9464-fdd4-491d-a0ce-51519a35486d","Type":"ContainerDied","Data":"cb92081d44e779c1995c55501d5f98901ddd3f07f0de92e04c6baceafaf829f9"}
Oct 30 05:31:12.840114 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:12.840083    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg" event={"ID":"02a30d07-33cc-4be0-a920-8e839f8a493b","Type":"ContainerStarted","Data":"6edbafc40219e80ff439ed3f7242858ca9865d902b2f26c5d68056ade57607ed"}
Oct 30 05:31:12.840194 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:12.840174    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg" event={"ID":"02a30d07-33cc-4be0-a920-8e839f8a493b","Type":"ContainerStarted","Data":"6711a50e52bf22a959f40c681519645490602863450dfcd1c5e9d767ee25d856"}
Oct 30 05:31:12.840241 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:12.840226    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg" event={"ID":"02a30d07-33cc-4be0-a920-8e839f8a493b","Type":"ContainerStarted","Data":"48404fd90fb71b783885237819f2f4910fd9a6406754484afa448bd4dd4c93db"}
Oct 30 05:31:12.840293 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:12.840281    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg" event={"ID":"02a30d07-33cc-4be0-a920-8e839f8a493b","Type":"ContainerStarted","Data":"08e26c7d02765ed34edd5f4fb785cf4ea1aec350f18adc0cc7f8db022e7fcbc8"}
Oct 30 05:31:12.840342 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:12.840331    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg" event={"ID":"02a30d07-33cc-4be0-a920-8e839f8a493b","Type":"ContainerStarted","Data":"9aed9118e145d26c4f2a156c26fb994908b975052e9f097fe175d72bb96c3f10"}
Oct 30 05:31:12.840390 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:12.840379    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg" event={"ID":"02a30d07-33cc-4be0-a920-8e839f8a493b","Type":"ContainerStarted","Data":"0f656eeb99e881bcfd502570bb7faec596f409051d357b13f35e2f28f80db074"}
Oct 30 05:31:13.269961 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:13.269867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:13.269961 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:13.269915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:13.270144 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:13.270026    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 05:31:13.270144 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:13.270026    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 05:31:13.270144 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:13.270089    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs podName:60bf362c-030f-4adf-9878-842b2fdc326b nodeName:}" failed. No retries permitted until 2026-10-30 05:31:29.270075386 +0000 UTC m=+34.208449743 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs") pod "network-metrics-daemon-pfdj6" (UID: "60bf362c-030f-4adf-9878-842b2fdc326b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 05:31:13.270144 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:13.270102    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret podName:7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa nodeName:}" failed. No retries permitted until 2026-10-30 05:31:29.270096518 +0000 UTC m=+34.208470881 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret") pod "global-pull-secret-syncer-kkcc6" (UID: "7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa") : object "kube-system"/"original-pull-secret" not registered
Oct 30 05:31:13.370366 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:13.370330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wkpm\" (UniqueName: \"kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm\") pod \"network-check-target-fdfg7\" (UID: \"f82afdb2-9cea-479b-8d0b-88a814886c3a\") " pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:13.370509 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:13.370460    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 05:31:13.370509 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:13.370476    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 05:31:13.370509 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:13.370485    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5wkpm for pod openshift-network-diagnostics/network-check-target-fdfg7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 05:31:13.370656 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:13.370531    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm podName:f82afdb2-9cea-479b-8d0b-88a814886c3a nodeName:}" failed. No retries permitted until 2026-10-30 05:31:29.370519315 +0000 UTC m=+34.308893671 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5wkpm" (UniqueName: "kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm") pod "network-check-target-fdfg7" (UID: "f82afdb2-9cea-479b-8d0b-88a814886c3a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 05:31:13.541390 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:13.541318    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:13.541390 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:13.541354    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:13.541602 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:13.541437    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:31:13.541602 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:13.541489    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:31:13.845342 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:13.845258    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6l5g" event={"ID":"e71a9464-fdd4-491d-a0ce-51519a35486d","Type":"ContainerStarted","Data":"83adf1f9f1edccc72b3e715dbf8a0a1238d659d7c8698c195c23bf037a53492f"}
Oct 30 05:31:13.890814 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:13.890766    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-g6l5g" podStartSLOduration=8.044855915 podStartE2EDuration="17.890751446s" totalImagesPullingTime="9.845895531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 05:30:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:13.890480975 +0000 UTC m=+18.828855353" watchObservedRunningTime="2026-10-30 05:31:13.890751446 +0000 UTC m=+18.829125825"
Oct 30 05:31:14.538129 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:14.538093    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:14.538316 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:14.538201    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:14.851012 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:14.850929    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg" event={"ID":"02a30d07-33cc-4be0-a920-8e839f8a493b","Type":"ContainerStarted","Data":"4f24b1cd9caeb25eadc0f3794337cd15f286464bec9d40005d803ed29e0cdf36"}
Oct 30 05:31:15.539036 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:15.539005    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:15.539175 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:15.539086    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:31:15.539235 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:15.539189    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:15.539314 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:15.539295    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:31:16.538380 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:16.538341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:16.538967 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:16.538483    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:16.859728 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:16.859685    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg" event={"ID":"02a30d07-33cc-4be0-a920-8e839f8a493b","Type":"ContainerStarted","Data":"7cd7e6cb4d6afe8d231a8f3d9c12075091b25b0035798043a6ad1f8886078ce9"}
Oct 30 05:31:16.859955 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:16.859918    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:16.875948 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:16.875920    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:16.946581 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:16.946495    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg" podStartSLOduration=6.457583481 podStartE2EDuration="20.946480847s" totalImagesPullingTime="14.488897366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 05:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:16.934472258 +0000 UTC m=+21.872846634" watchObservedRunningTime="2026-10-30 05:31:16.946480847 +0000 UTC m=+21.884855225"
Oct 30 05:31:17.538218 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:17.538180    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:17.538453 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:17.538321    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:31:17.538453 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:17.538359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:17.539057 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:17.538450    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:31:17.861773 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:17.861702    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 05:31:17.862130 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:17.862112    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:17.878288 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:17.878266    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:18.538180 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:18.538147    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:18.538359 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:18.538292    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:18.863605 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:18.863378    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 05:31:18.908323 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:18.908296    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kkcc6"]
Oct 30 05:31:18.908464 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:18.908397    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:18.908503 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:18.908477    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:31:18.911296 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:18.911264    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fdfg7"]
Oct 30 05:31:18.911412 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:18.911379    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:18.911502 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:18.911481    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:31:18.911916 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:18.911888    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pfdj6"]
Oct 30 05:31:18.912022 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:18.911984    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:18.912100 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:18.912080    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:19.865488 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:19.865458    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 05:31:20.537901 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:20.537861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:20.538092 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:20.537924    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:20.538092 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:20.538019    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:20.538092 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:20.538045    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:20.538264 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:20.538096    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:31:20.538264 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:20.538137    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:31:20.855784 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:20.855748    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:22.537701 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:22.537668    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:22.538096 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:22.537769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:22.538096 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:22.537783    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkcc6" podUID="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa"
Oct 30 05:31:22.538096 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:22.537794    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:22.538096 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:22.537845    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fdfg7" podUID="f82afdb2-9cea-479b-8d0b-88a814886c3a"
Oct 30 05:31:22.538096 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:22.537934    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfdj6" podUID="60bf362c-030f-4adf-9878-842b2fdc326b"
Oct 30 05:31:24.360280 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.360207    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeReady"
Oct 30 05:31:24.360638 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.360327    2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 05:31:24.414145 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.414111    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b668646-vshtx"]
Oct 30 05:31:24.416725 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.416705    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-gh627"]
Oct 30 05:31:24.416864 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.416838    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.419265 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.419245    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d65554659-bdvbl"]
Oct 30 05:31:24.419389 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.419372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-gh627"
Oct 30 05:31:24.421871 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.421855    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.426352 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.426330    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-k5d59"]
Oct 30 05:31:24.429629 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.429611    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"]
Oct 30 05:31:24.429710 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.429664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-k5d59"
Oct 30 05:31:24.432289 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.432273    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-59857694c5-wq9fj"]
Oct 30 05:31:24.432405 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.432390    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:24.435212 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.435196    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-4sd8r"]
Oct 30 05:31:24.435321 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.435307    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.438361 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.438341    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2msqk\""
Oct 30 05:31:24.438765 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.438750    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"]
Oct 30 05:31:24.438907 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.438885    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-4sd8r"
Oct 30 05:31:24.441348 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.441332    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-swtbv"]
Oct 30 05:31:24.441453 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.441440    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:24.444103 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.444082    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 05:31:24.444200 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.444163    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"]
Oct 30 05:31:24.444403 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.444199    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 05:31:24.444403 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.444350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.446964 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.446948    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq"]
Oct 30 05:31:24.449377 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.447345    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:24.455711 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.455688    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"]
Oct 30 05:31:24.455858 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.455830    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq"
Oct 30 05:31:24.458783 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.458761    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-58dpm"]
Oct 30 05:31:24.458880 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.458808    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 05:31:24.458988 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.458969    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:24.461813 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.461794    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf"]
Oct 30 05:31:24.461920 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.461905    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.467788 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.467665    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"]
Oct 30 05:31:24.467788 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.467780    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf"
Oct 30 05:31:24.472981 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.472960    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bzbfz"]
Oct 30 05:31:24.473101 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.473067    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:24.479406 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.479378    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mfzzm"]
Oct 30 05:31:24.479522 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.479467    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:24.482446 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.482425    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b668646-vshtx"]
Oct 30 05:31:24.482559 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.482459    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-k5d59"]
Oct 30 05:31:24.482559 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.482482    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"]
Oct 30 05:31:24.482559 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.482502    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-gh627"]
Oct 30 05:31:24.482559 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.482523    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d65554659-bdvbl"]
Oct 30 05:31:24.482559 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.482543    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"]
Oct 30 05:31:24.482559 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.482554    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mfzzm"
Oct 30 05:31:24.482559 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.482563    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-swtbv"]
Oct 30 05:31:24.482856 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.482582    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-58dpm"]
Oct 30 05:31:24.482856 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.482621    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq"]
Oct 30 05:31:24.482856 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.482641    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59857694c5-wq9fj"]
Oct 30 05:31:24.482856 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.482658    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-4sd8r"]
Oct 30 05:31:24.485769 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.485750    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"]
Oct 30 05:31:24.488610 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.486965    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mfzzm"]
Oct 30 05:31:24.488610 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.487438    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bzbfz"]
Oct 30 05:31:24.498557 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.498534    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf"]
Oct 30 05:31:24.501998 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.501406    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"]
Oct 30 05:31:24.502126 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.502106    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"]
Oct 30 05:31:24.538019 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.537993    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:24.538152 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.538091    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:24.538152 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.538114    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:24.539139 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.538956    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 05:31:24.539311 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539294    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 05:31:24.539368 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539309    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 05:31:24.539413 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539387    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 05:31:24.539464 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539426    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 05:31:24.539745 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539730    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 05:31:24.539859 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539732    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 05:31:24.539859 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539744    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 05:31:24.539859 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539765    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 05:31:24.540065 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539786    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 05:31:24.540065 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539798    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 05:31:24.540065 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539900    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 05:31:24.540065 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539926    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 05:31:24.540065 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.539953    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-7pqs8\""
Oct 30 05:31:24.540065 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.540008    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 05:31:24.540065 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.540063    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-cxh97\""
Oct 30 05:31:24.540262 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.540156    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 05:31:24.540262 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.540214    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jvn8d\""
Oct 30 05:31:24.543314 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfzsh\" (UniqueName: \"kubernetes.io/projected/86fadfbe-f000-4b42-a8ff-7b67061959c5-kube-api-access-jfzsh\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.543404 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab824d2b-de9a-4384-a4a8-166f58808be8-config\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.543404 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d4vl\" (UniqueName: \"kubernetes.io/projected/dfbf26d6-b6c8-4d33-9219-5e9fff25a103-kube-api-access-8d4vl\") pod \"network-check-source-6f78d59f99-dvtwf\" (UID: \"dfbf26d6-b6c8-4d33-9219-5e9fff25a103\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf"
Oct 30 05:31:24.543476 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-default-certificate\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.543538 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543517    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dac2ba5d-8990-47e3-aa65-02ff7500ac01-ca-trust-extracted\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.543571 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m7s7\" (UniqueName: \"kubernetes.io/projected/fcdb9d1a-474b-49f3-8683-f3874ad9d769-kube-api-access-5m7s7\") pod \"kas-connection-checker-685895c7c9-gh627\" (UID: \"fcdb9d1a-474b-49f3-8683-f3874ad9d769\") " pod="kube-system/kas-connection-checker-685895c7c9-gh627"
Oct 30 05:31:24.543660 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7pkr\" (UniqueName: \"kubernetes.io/projected/666a74c6-3979-480a-b4d7-4872c84b2413-kube-api-access-n7pkr\") pod \"kas-connection-checker-685895c7c9-k5d59\" (UID: \"666a74c6-3979-480a-b4d7-4872c84b2413\") " pod="kube-system/kas-connection-checker-685895c7c9-k5d59"
Oct 30 05:31:24.543698 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:24.543698 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543690    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrchp\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-kube-api-access-wrchp\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.543765 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.543765 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.543821 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/86fadfbe-f000-4b42-a8ff-7b67061959c5-snapshots\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.543852 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bptqj\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-kube-api-access-bptqj\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.543899 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543883    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0cf86020-761f-4fbc-808e-7dc6c5d5515f-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:24.543977 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543916    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86fadfbe-f000-4b42-a8ff-7b67061959c5-service-ca-bundle\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.543977 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:24.543977 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.543966    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-bound-sa-token\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.544091 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544019    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dffbj\" (UniqueName: \"kubernetes.io/projected/4684e0e6-736f-41bf-8f22-77245e339d58-kube-api-access-dffbj\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.544091 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.544091 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-certificates\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.544190 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv985\" (UniqueName: \"kubernetes.io/projected/0cf86020-761f-4fbc-808e-7dc6c5d5515f-kube-api-access-sv985\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:24.544190 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-installation-pull-secrets\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.544272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-image-registry-private-configuration\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.544272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-certificates\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.544272 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86fadfbe-f000-4b42-a8ff-7b67061959c5-serving-cert\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.544400 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdz8s\" (UniqueName: \"kubernetes.io/projected/ab824d2b-de9a-4384-a4a8-166f58808be8-kube-api-access-xdz8s\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.544400 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:24.544400 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41edacc9-dea1-4343-96f5-5caf8a82fe84-serving-cert\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:24.544544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a655cfec-4ee3-4ac9-badc-4fb318603506-cert\") pod \"ingress-canary-mfzzm\" (UID: \"a655cfec-4ee3-4ac9-badc-4fb318603506\") " pod="openshift-ingress-canary/ingress-canary-mfzzm"
Oct 30 05:31:24.544544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544462    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c46db00b-3c48-4d8d-844c-918d8126cc41-config-volume\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:24.544544 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c46db00b-3c48-4d8d-844c-918d8126cc41-tmp-dir\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:24.544700 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-installation-pull-secrets\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.544700 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-nginx-conf\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:24.544700 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvb6d\" (UniqueName: \"kubernetes.io/projected/46c256fd-a89b-41f9-95aa-d8e81f783820-kube-api-access-rvb6d\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:24.544700 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544689    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab824d2b-de9a-4384-a4a8-166f58808be8-trusted-ca\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.544853 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:24.544853 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-image-registry-private-configuration\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.544853 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xdbt\" (UniqueName: \"kubernetes.io/projected/c46db00b-3c48-4d8d-844c-918d8126cc41-kube-api-access-7xdbt\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:24.545001 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.545001 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544928    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 05:31:24.545001 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct475\" (UniqueName: \"kubernetes.io/projected/a0ea8679-d95b-49d6-a614-9bb4eb4ae029-kube-api-access-ct475\") pod \"kas-connection-checker-685895c7c9-4sd8r\" (UID: \"a0ea8679-d95b-49d6-a614-9bb4eb4ae029\") " pod="kube-system/kas-connection-checker-685895c7c9-4sd8r"
Oct 30 05:31:24.545001 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544952    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 05:31:24.545001 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544967    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 05:31:24.545001 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544982    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 05:31:24.545001 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.544983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-trusted-ca\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.545308 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf86020-761f-4fbc-808e-7dc6c5d5515f-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:24.545308 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-ca-trust-extracted\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.545308 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545035    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 05:31:24.545308 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:24.545308 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86fadfbe-f000-4b42-a8ff-7b67061959c5-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.545308 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545200    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 05:31:24.545308 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-stats-auth\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.545308 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545238    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86fadfbe-f000-4b42-a8ff-7b67061959c5-tmp\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.545308 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545265    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-bound-sa-token\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.545308 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545283    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:24.545308 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml8s5\" (UniqueName: \"kubernetes.io/projected/2f92fa2c-9133-484f-b326-a12b2b618900-kube-api-access-ml8s5\") pod \"volume-data-source-validator-58c9f5f46d-gshpq\" (UID: \"2f92fa2c-9133-484f-b326-a12b2b618900\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq"
Oct 30 05:31:24.545308 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545317    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545291    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545336    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab824d2b-de9a-4384-a4a8-166f58808be8-serving-cert\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545352    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tmzj\" (UniqueName: \"kubernetes.io/projected/41edacc9-dea1-4343-96f5-5caf8a82fe84-kube-api-access-9tmzj\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545410    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545433    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-trusted-ca\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545420    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545489    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545551    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41edacc9-dea1-4343-96f5-5caf8a82fe84-config\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq7d9\" (UniqueName: \"kubernetes.io/projected/a655cfec-4ee3-4ac9-badc-4fb318603506-kube-api-access-pq7d9\") pod \"ingress-canary-mfzzm\" (UID: \"a655cfec-4ee3-4ac9-badc-4fb318603506\") " pod="openshift-ingress-canary/ingress-canary-mfzzm"
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545656    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l72vm\" (UniqueName: \"kubernetes.io/projected/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-kube-api-access-l72vm\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545715    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545731    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545740    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545771    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545801    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545772    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545824    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 05:31:24.545844 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545828    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 05:31:24.546633 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545866    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:24.546633 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545864    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 05:31:24.546633 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.545873    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:24.546633 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.546150    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 05:31:24.546633 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.546280    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 05:31:24.546633 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.546374    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 05:31:24.556925 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.556906    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 05:31:24.557074 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.557058    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 05:31:24.639105 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.639012    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 05:31:24.639105 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.639012    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 05:31:24.639105 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.639048    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wks87\""
Oct 30 05:31:24.639105 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.639087    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-nr89r\""
Oct 30 05:31:24.639336 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.639135    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v8jj4\""
Oct 30 05:31:24.646490 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-trusted-ca\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.646611 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41edacc9-dea1-4343-96f5-5caf8a82fe84-config\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:24.646611 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq7d9\" (UniqueName: \"kubernetes.io/projected/a655cfec-4ee3-4ac9-badc-4fb318603506-kube-api-access-pq7d9\") pod \"ingress-canary-mfzzm\" (UID: \"a655cfec-4ee3-4ac9-badc-4fb318603506\") " pod="openshift-ingress-canary/ingress-canary-mfzzm"
Oct 30 05:31:24.646611 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l72vm\" (UniqueName: \"kubernetes.io/projected/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-kube-api-access-l72vm\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:24.646761 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:24.646761 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfzsh\" (UniqueName: \"kubernetes.io/projected/86fadfbe-f000-4b42-a8ff-7b67061959c5-kube-api-access-jfzsh\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.646761 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab824d2b-de9a-4384-a4a8-166f58808be8-config\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.646761 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.646722    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 05:31:24.646761 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d4vl\" (UniqueName: \"kubernetes.io/projected/dfbf26d6-b6c8-4d33-9219-5e9fff25a103-kube-api-access-8d4vl\") pod \"network-check-source-6f78d59f99-dvtwf\" (UID: \"dfbf26d6-b6c8-4d33-9219-5e9fff25a103\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf"
Oct 30 05:31:24.646761 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-default-certificate\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.647042 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.646778    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls podName:c46db00b-3c48-4d8d-844c-918d8126cc41 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:25.146760431 +0000 UTC m=+30.085134806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls") pod "dns-default-bzbfz" (UID: "c46db00b-3c48-4d8d-844c-918d8126cc41") : secret "dns-default-metrics-tls" not found
Oct 30 05:31:24.647042 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646798    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dac2ba5d-8990-47e3-aa65-02ff7500ac01-ca-trust-extracted\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.647042 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m7s7\" (UniqueName: \"kubernetes.io/projected/fcdb9d1a-474b-49f3-8683-f3874ad9d769-kube-api-access-5m7s7\") pod \"kas-connection-checker-685895c7c9-gh627\" (UID: \"fcdb9d1a-474b-49f3-8683-f3874ad9d769\") " pod="kube-system/kas-connection-checker-685895c7c9-gh627"
Oct 30 05:31:24.647042 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7pkr\" (UniqueName: \"kubernetes.io/projected/666a74c6-3979-480a-b4d7-4872c84b2413-kube-api-access-n7pkr\") pod \"kas-connection-checker-685895c7c9-k5d59\" (UID: \"666a74c6-3979-480a-b4d7-4872c84b2413\") " pod="kube-system/kas-connection-checker-685895c7c9-k5d59"
Oct 30 05:31:24.647042 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:24.647042 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrchp\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-kube-api-access-wrchp\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.647042 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.647042 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.647042 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.646989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/86fadfbe-f000-4b42-a8ff-7b67061959c5-snapshots\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.647042 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bptqj\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-kube-api-access-bptqj\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0cf86020-761f-4fbc-808e-7dc6c5d5515f-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86fadfbe-f000-4b42-a8ff-7b67061959c5-service-ca-bundle\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-bound-sa-token\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dffbj\" (UniqueName: \"kubernetes.io/projected/4684e0e6-736f-41bf-8f22-77245e339d58-kube-api-access-dffbj\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-certificates\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv985\" (UniqueName: \"kubernetes.io/projected/0cf86020-761f-4fbc-808e-7dc6c5d5515f-kube-api-access-sv985\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647267    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41edacc9-dea1-4343-96f5-5caf8a82fe84-config\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-installation-pull-secrets\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-image-registry-private-configuration\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-certificates\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86fadfbe-f000-4b42-a8ff-7b67061959c5-serving-cert\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdz8s\" (UniqueName: \"kubernetes.io/projected/ab824d2b-de9a-4384-a4a8-166f58808be8-kube-api-access-xdz8s\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.647507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41edacc9-dea1-4343-96f5-5caf8a82fe84-serving-cert\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647473    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dac2ba5d-8990-47e3-aa65-02ff7500ac01-ca-trust-extracted\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a655cfec-4ee3-4ac9-badc-4fb318603506-cert\") pod \"ingress-canary-mfzzm\" (UID: \"a655cfec-4ee3-4ac9-badc-4fb318603506\") " pod="openshift-ingress-canary/ingress-canary-mfzzm"
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.647614    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle podName:4684e0e6-736f-41bf-8f22-77245e339d58 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:25.147572623 +0000 UTC m=+30.085946980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle") pod "router-default-59857694c5-wq9fj" (UID: "4684e0e6-736f-41bf-8f22-77245e339d58") : configmap references non-existent config key: service-ca.crt
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647656    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab824d2b-de9a-4384-a4a8-166f58808be8-config\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647656    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-trusted-ca\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.647866    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0cf86020-761f-4fbc-808e-7dc6c5d5515f-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.647959    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.648003    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs podName:4684e0e6-736f-41bf-8f22-77245e339d58 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:25.147988505 +0000 UTC m=+30.086362881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs") pod "router-default-59857694c5-wq9fj" (UID: "4684e0e6-736f-41bf-8f22-77245e339d58") : secret "router-metrics-certs-default" not found
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.648025    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648040    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86fadfbe-f000-4b42-a8ff-7b67061959c5-service-ca-bundle\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c46db00b-3c48-4d8d-844c-918d8126cc41-config-volume\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.648102    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert podName:b0bdcd7b-f471-405a-87b2-ebedb8a1c950 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:25.148084545 +0000 UTC m=+30.086458922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v6b7k" (UID: "b0bdcd7b-f471-405a-87b2-ebedb8a1c950") : secret "networking-console-plugin-cert" not found
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c46db00b-3c48-4d8d-844c-918d8126cc41-tmp-dir\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:24.648176 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-installation-pull-secrets\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648183    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-nginx-conf\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvb6d\" (UniqueName: \"kubernetes.io/projected/46c256fd-a89b-41f9-95aa-d8e81f783820-kube-api-access-rvb6d\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab824d2b-de9a-4384-a4a8-166f58808be8-trusted-ca\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-image-registry-private-configuration\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdbt\" (UniqueName: \"kubernetes.io/projected/c46db00b-3c48-4d8d-844c-918d8126cc41-kube-api-access-7xdbt\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct475\" (UniqueName: \"kubernetes.io/projected/a0ea8679-d95b-49d6-a614-9bb4eb4ae029-kube-api-access-ct475\") pod \"kas-connection-checker-685895c7c9-4sd8r\" (UID: \"a0ea8679-d95b-49d6-a614-9bb4eb4ae029\") " pod="kube-system/kas-connection-checker-685895c7c9-4sd8r"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-trusted-ca\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf86020-761f-4fbc-808e-7dc6c5d5515f-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648652    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c46db00b-3c48-4d8d-844c-918d8126cc41-tmp-dir\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-ca-trust-extracted\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648678    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/86fadfbe-f000-4b42-a8ff-7b67061959c5-snapshots\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:24.648802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86fadfbe-f000-4b42-a8ff-7b67061959c5-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.649717 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-stats-auth\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.649717 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86fadfbe-f000-4b42-a8ff-7b67061959c5-tmp\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.649717 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-bound-sa-token\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.649717 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml8s5\" (UniqueName: \"kubernetes.io/projected/2f92fa2c-9133-484f-b326-a12b2b618900-kube-api-access-ml8s5\") pod \"volume-data-source-validator-58c9f5f46d-gshpq\" (UID: \"2f92fa2c-9133-484f-b326-a12b2b618900\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq"
Oct 30 05:31:24.649717 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab824d2b-de9a-4384-a4a8-166f58808be8-serving-cert\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.649717 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.649038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tmzj\" (UniqueName: \"kubernetes.io/projected/41edacc9-dea1-4343-96f5-5caf8a82fe84-kube-api-access-9tmzj\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:24.649717 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.649133    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-ca-trust-extracted\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.649717 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.649223    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 05:31:24.649717 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.649235    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d65554659-bdvbl: secret "image-registry-tls" not found
Oct 30 05:31:24.649717 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.649272    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls podName:e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a nodeName:}" failed. No retries permitted until 2026-10-30 05:31:25.149259682 +0000 UTC m=+30.087634041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls") pod "image-registry-7d65554659-bdvbl" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a") : secret "image-registry-tls" not found
Oct 30 05:31:24.649717 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.649455    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-nginx-conf\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:24.650242 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.649751    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 05:31:24.650242 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.649785    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b668646-vshtx: secret "image-registry-tls" not found
Oct 30 05:31:24.650242 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.649824    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls podName:dac2ba5d-8990-47e3-aa65-02ff7500ac01 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:25.149810611 +0000 UTC m=+30.088184970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls") pod "image-registry-6b668646-vshtx" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01") : secret "image-registry-tls" not found
Oct 30 05:31:24.650381 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.650346    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-certificates\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.651144 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.651120    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86fadfbe-f000-4b42-a8ff-7b67061959c5-tmp\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.651390 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.651362    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86fadfbe-f000-4b42-a8ff-7b67061959c5-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.652274 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.652248    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86fadfbe-f000-4b42-a8ff-7b67061959c5-serving-cert\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.652360 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.652289    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-certificates\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.652614 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.652561    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-trusted-ca\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.652614 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.648625    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c46db00b-3c48-4d8d-844c-918d8126cc41-config-volume\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:24.652744 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.652680    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 05:31:24.652744 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.652708    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a655cfec-4ee3-4ac9-badc-4fb318603506-cert\") pod \"ingress-canary-mfzzm\" (UID: \"a655cfec-4ee3-4ac9-badc-4fb318603506\") " pod="openshift-ingress-canary/ingress-canary-mfzzm"
Oct 30 05:31:24.652744 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.652722    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls podName:bfa3e142-50e3-4b85-83fa-46f7dc8b7858 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:25.152708197 +0000 UTC m=+30.091082591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6xnts" (UID: "bfa3e142-50e3-4b85-83fa-46f7dc8b7858") : secret "cluster-monitoring-operator-tls" not found
Oct 30 05:31:24.653091 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.653066    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 05:31:24.653568 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.653542    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-default-certificate\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.653674 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.653639    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:24.653909 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.653880    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41edacc9-dea1-4343-96f5-5caf8a82fe84-serving-cert\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:24.654051 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.654030    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-stats-auth\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.654181 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.654147    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-installation-pull-secrets\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.654322 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.654301    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab824d2b-de9a-4384-a4a8-166f58808be8-serving-cert\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.654616 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.654579    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf86020-761f-4fbc-808e-7dc6c5d5515f-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:24.654678 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.654655    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab824d2b-de9a-4384-a4a8-166f58808be8-trusted-ca\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.654754 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:24.654741    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls podName:46c256fd-a89b-41f9-95aa-d8e81f783820 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:25.154725351 +0000 UTC m=+30.093099732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-qsjvb" (UID: "46c256fd-a89b-41f9-95aa-d8e81f783820") : secret "samples-operator-tls" not found
Oct 30 05:31:24.655220 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.655199    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-image-registry-private-configuration\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.655283 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.655204    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-image-registry-private-configuration\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.655566 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.655549    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-installation-pull-secrets\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.672120 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.672092    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfzsh\" (UniqueName: \"kubernetes.io/projected/86fadfbe-f000-4b42-a8ff-7b67061959c5-kube-api-access-jfzsh\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.672216 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.672134    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l72vm\" (UniqueName: \"kubernetes.io/projected/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-kube-api-access-l72vm\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:24.672216 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.672148    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d4vl\" (UniqueName: \"kubernetes.io/projected/dfbf26d6-b6c8-4d33-9219-5e9fff25a103-kube-api-access-8d4vl\") pod \"network-check-source-6f78d59f99-dvtwf\" (UID: \"dfbf26d6-b6c8-4d33-9219-5e9fff25a103\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf"
Oct 30 05:31:24.672216 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.672175    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7pkr\" (UniqueName: \"kubernetes.io/projected/666a74c6-3979-480a-b4d7-4872c84b2413-kube-api-access-n7pkr\") pod \"kas-connection-checker-685895c7c9-k5d59\" (UID: \"666a74c6-3979-480a-b4d7-4872c84b2413\") " pod="kube-system/kas-connection-checker-685895c7c9-k5d59"
Oct 30 05:31:24.672582 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.672565    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq7d9\" (UniqueName: \"kubernetes.io/projected/a655cfec-4ee3-4ac9-badc-4fb318603506-kube-api-access-pq7d9\") pod \"ingress-canary-mfzzm\" (UID: \"a655cfec-4ee3-4ac9-badc-4fb318603506\") " pod="openshift-ingress-canary/ingress-canary-mfzzm"
Oct 30 05:31:24.677474 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.677453    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bptqj\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-kube-api-access-bptqj\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.680522 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.680492    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct475\" (UniqueName: \"kubernetes.io/projected/a0ea8679-d95b-49d6-a614-9bb4eb4ae029-kube-api-access-ct475\") pod \"kas-connection-checker-685895c7c9-4sd8r\" (UID: \"a0ea8679-d95b-49d6-a614-9bb4eb4ae029\") " pod="kube-system/kas-connection-checker-685895c7c9-4sd8r"
Oct 30 05:31:24.680734 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.680713    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xdbt\" (UniqueName: \"kubernetes.io/projected/c46db00b-3c48-4d8d-844c-918d8126cc41-kube-api-access-7xdbt\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:24.681288 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.681263    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrchp\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-kube-api-access-wrchp\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.682511 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.682466    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdz8s\" (UniqueName: \"kubernetes.io/projected/ab824d2b-de9a-4384-a4a8-166f58808be8-kube-api-access-xdz8s\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.682632 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.682547    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m7s7\" (UniqueName: \"kubernetes.io/projected/fcdb9d1a-474b-49f3-8683-f3874ad9d769-kube-api-access-5m7s7\") pod \"kas-connection-checker-685895c7c9-gh627\" (UID: \"fcdb9d1a-474b-49f3-8683-f3874ad9d769\") " pod="kube-system/kas-connection-checker-685895c7c9-gh627"
Oct 30 05:31:24.683059 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.683031    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-bound-sa-token\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:24.683977 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.683956    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-bound-sa-token\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:24.684958 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.684918    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvb6d\" (UniqueName: \"kubernetes.io/projected/46c256fd-a89b-41f9-95aa-d8e81f783820-kube-api-access-rvb6d\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:24.686113 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.686091    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv985\" (UniqueName: \"kubernetes.io/projected/0cf86020-761f-4fbc-808e-7dc6c5d5515f-kube-api-access-sv985\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:24.687963 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.687937    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tmzj\" (UniqueName: \"kubernetes.io/projected/41edacc9-dea1-4343-96f5-5caf8a82fe84-kube-api-access-9tmzj\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:24.689221 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.689168    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml8s5\" (UniqueName: \"kubernetes.io/projected/2f92fa2c-9133-484f-b326-a12b2b618900-kube-api-access-ml8s5\") pod \"volume-data-source-validator-58c9f5f46d-gshpq\" (UID: \"2f92fa2c-9133-484f-b326-a12b2b618900\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq"
Oct 30 05:31:24.689375 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.689357    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dffbj\" (UniqueName: \"kubernetes.io/projected/4684e0e6-736f-41bf-8f22-77245e339d58-kube-api-access-dffbj\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:24.737688 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.737658    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-gh627"
Oct 30 05:31:24.751491 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.751356    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-k5d59"
Oct 30 05:31:24.768677 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.768648    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:24.782414 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.782387    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-4sd8r"
Oct 30 05:31:24.791936 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.791732    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:24.801629 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.801435    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:24.814963 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.813949    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq"
Oct 30 05:31:24.825656 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.825262    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:24.834779 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.834047    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf"
Oct 30 05:31:24.854000 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:24.853944    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mfzzm"
Oct 30 05:31:25.056542 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.056518    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-gh627"]
Oct 30 05:31:25.057637 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.057607    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-k5d59"]
Oct 30 05:31:25.059275 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:25.059252    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod666a74c6_3979_480a_b4d7_4872c84b2413.slice/crio-8086d04205d43e965e252adecc780e5d80289a8081c06ab5e9c6436a3e51457a WatchSource:0}: Error finding container 8086d04205d43e965e252adecc780e5d80289a8081c06ab5e9c6436a3e51457a: Status 404 returned error can't find the container with id 8086d04205d43e965e252adecc780e5d80289a8081c06ab5e9c6436a3e51457a
Oct 30 05:31:25.059848 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:25.059765    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcdb9d1a_474b_49f3_8683_f3874ad9d769.slice/crio-2ebca268a5ecc2180eb5e0c3d08af50a88603c4550cf29243ada2b9f338f6567 WatchSource:0}: Error finding container 2ebca268a5ecc2180eb5e0c3d08af50a88603c4550cf29243ada2b9f338f6567: Status 404 returned error can't find the container with id 2ebca268a5ecc2180eb5e0c3d08af50a88603c4550cf29243ada2b9f338f6567
Oct 30 05:31:25.089037 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.089013    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"]
Oct 30 05:31:25.091020 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:25.090991    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41edacc9_dea1_4343_96f5_5caf8a82fe84.slice/crio-f367b6ce0cb23d3c0cb99db8f880c07c8359f3bc27fd593149ba7f2d801f3ca4 WatchSource:0}: Error finding container f367b6ce0cb23d3c0cb99db8f880c07c8359f3bc27fd593149ba7f2d801f3ca4: Status 404 returned error can't find the container with id f367b6ce0cb23d3c0cb99db8f880c07c8359f3bc27fd593149ba7f2d801f3ca4
Oct 30 05:31:25.118056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.117971    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-4sd8r"]
Oct 30 05:31:25.123824 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:25.123255    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0ea8679_d95b_49d6_a614_9bb4eb4ae029.slice/crio-ca318e520dc7f05fc01002867cc54d58bc9e6382ea5e0c6ada15c4861c8f8ee0 WatchSource:0}: Error finding container ca318e520dc7f05fc01002867cc54d58bc9e6382ea5e0c6ada15c4861c8f8ee0: Status 404 returned error can't find the container with id ca318e520dc7f05fc01002867cc54d58bc9e6382ea5e0c6ada15c4861c8f8ee0
Oct 30 05:31:25.159253 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.159223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:25.159680 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.159260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:25.159680 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.159299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:25.159680 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.159338    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:25.159680 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.159376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:25.159680 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.159419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:25.159680 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.159450    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:25.159680 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.159536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.159694    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.159757    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls podName:c46db00b-3c48-4d8d-844c-918d8126cc41 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:26.159738169 +0000 UTC m=+31.098112550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls") pod "dns-default-bzbfz" (UID: "c46db00b-3c48-4d8d-844c-918d8126cc41") : secret "dns-default-metrics-tls" not found
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160418    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle podName:4684e0e6-736f-41bf-8f22-77245e339d58 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:26.160400942 +0000 UTC m=+31.098775312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle") pod "router-default-59857694c5-wq9fj" (UID: "4684e0e6-736f-41bf-8f22-77245e339d58") : configmap references non-existent config key: service-ca.crt
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160493    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160530    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs podName:4684e0e6-736f-41bf-8f22-77245e339d58 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:26.16051881 +0000 UTC m=+31.098893183 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs") pod "router-default-59857694c5-wq9fj" (UID: "4684e0e6-736f-41bf-8f22-77245e339d58") : secret "router-metrics-certs-default" not found
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160604    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160637    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert podName:b0bdcd7b-f471-405a-87b2-ebedb8a1c950 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:26.160626912 +0000 UTC m=+31.099001273 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v6b7k" (UID: "b0bdcd7b-f471-405a-87b2-ebedb8a1c950") : secret "networking-console-plugin-cert" not found
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160700    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160713    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b668646-vshtx: secret "image-registry-tls" not found
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160745    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls podName:dac2ba5d-8990-47e3-aa65-02ff7500ac01 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:26.160735252 +0000 UTC m=+31.099109622 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls") pod "image-registry-6b668646-vshtx" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01") : secret "image-registry-tls" not found
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160800    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160832    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls podName:bfa3e142-50e3-4b85-83fa-46f7dc8b7858 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:26.160821844 +0000 UTC m=+31.099196201 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6xnts" (UID: "bfa3e142-50e3-4b85-83fa-46f7dc8b7858") : secret "cluster-monitoring-operator-tls" not found
Oct 30 05:31:25.160868 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160870    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 05:31:25.161401 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160907    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls podName:46c256fd-a89b-41f9-95aa-d8e81f783820 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:26.160888298 +0000 UTC m=+31.099262670 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-qsjvb" (UID: "46c256fd-a89b-41f9-95aa-d8e81f783820") : secret "samples-operator-tls" not found
Oct 30 05:31:25.161401 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160965    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 05:31:25.161401 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.160975    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d65554659-bdvbl: secret "image-registry-tls" not found
Oct 30 05:31:25.161401 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:25.161003    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls podName:e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a nodeName:}" failed. No retries permitted until 2026-10-30 05:31:26.160994554 +0000 UTC m=+31.099368925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls") pod "image-registry-7d65554659-bdvbl" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a") : secret "image-registry-tls" not found
Oct 30 05:31:25.164547 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.163933    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq"]
Oct 30 05:31:25.166143 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:25.166113    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f92fa2c_9133_484f_b326_a12b2b618900.slice/crio-f99c68ea19609ad8745897cfdc054bbd985b7c73aa36064660d0e28411f36e63 WatchSource:0}: Error finding container f99c68ea19609ad8745897cfdc054bbd985b7c73aa36064660d0e28411f36e63: Status 404 returned error can't find the container with id f99c68ea19609ad8745897cfdc054bbd985b7c73aa36064660d0e28411f36e63
Oct 30 05:31:25.182369 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.181404    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-swtbv"]
Oct 30 05:31:25.192284 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:25.192105    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86fadfbe_f000_4b42_a8ff_7b67061959c5.slice/crio-09fa66e9e0d8ffed1d2f7c8685697159fb6de7aac8500a5df10dfba45090ad1f WatchSource:0}: Error finding container 09fa66e9e0d8ffed1d2f7c8685697159fb6de7aac8500a5df10dfba45090ad1f: Status 404 returned error can't find the container with id 09fa66e9e0d8ffed1d2f7c8685697159fb6de7aac8500a5df10dfba45090ad1f
Oct 30 05:31:25.199994 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.199971    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"]
Oct 30 05:31:25.206775 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:25.206747    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf86020_761f_4fbc_808e_7dc6c5d5515f.slice/crio-50eb74303b4645b568e99c96ad8fbeaa01b9f8cc49e117a8b391ccf8104207ad WatchSource:0}: Error finding container 50eb74303b4645b568e99c96ad8fbeaa01b9f8cc49e117a8b391ccf8104207ad: Status 404 returned error can't find the container with id 50eb74303b4645b568e99c96ad8fbeaa01b9f8cc49e117a8b391ccf8104207ad
Oct 30 05:31:25.221656 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.220491    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mfzzm"]
Oct 30 05:31:25.222736 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.222716    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-58dpm"]
Oct 30 05:31:25.226505 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:25.226058    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda655cfec_4ee3_4ac9_badc_4fb318603506.slice/crio-8e306019d1f5901620e7be039d572cd059f104094e542380c8ba3f76b2bd0133 WatchSource:0}: Error finding container 8e306019d1f5901620e7be039d572cd059f104094e542380c8ba3f76b2bd0133: Status 404 returned error can't find the container with id 8e306019d1f5901620e7be039d572cd059f104094e542380c8ba3f76b2bd0133
Oct 30 05:31:25.228339 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.227773    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf"]
Oct 30 05:31:25.230540 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:25.230519    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab824d2b_de9a_4384_a4a8_166f58808be8.slice/crio-f3d5ba1a086c66afec5f3dfe4bbbc0781dabebdf88b9126df4a0dff0a7794d8a WatchSource:0}: Error finding container f3d5ba1a086c66afec5f3dfe4bbbc0781dabebdf88b9126df4a0dff0a7794d8a: Status 404 returned error can't find the container with id f3d5ba1a086c66afec5f3dfe4bbbc0781dabebdf88b9126df4a0dff0a7794d8a
Oct 30 05:31:25.242646 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.242627    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:31:25.242784 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.242770    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 05:31:25.243216 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.243170    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:31:25.464913 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.464794    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:25.691511 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.691462    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:25.890599 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.890350    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-swtbv" event={"ID":"86fadfbe-f000-4b42-a8ff-7b67061959c5","Type":"ContainerStarted","Data":"09fa66e9e0d8ffed1d2f7c8685697159fb6de7aac8500a5df10dfba45090ad1f"}
Oct 30 05:31:25.892018 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.891939    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq" event={"ID":"2f92fa2c-9133-484f-b326-a12b2b618900","Type":"ContainerStarted","Data":"f99c68ea19609ad8745897cfdc054bbd985b7c73aa36064660d0e28411f36e63"}
Oct 30 05:31:25.894439 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.893507    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-k5d59" event={"ID":"666a74c6-3979-480a-b4d7-4872c84b2413","Type":"ContainerStarted","Data":"4ca31a5b000ba5feaf42b7261aa7419eea98c3fcd3e22e2a4f3db07da6ebcb94"}
Oct 30 05:31:25.894439 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.893547    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-k5d59" event={"ID":"666a74c6-3979-480a-b4d7-4872c84b2413","Type":"ContainerStarted","Data":"8086d04205d43e965e252adecc780e5d80289a8081c06ab5e9c6436a3e51457a"}
Oct 30 05:31:25.896964 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.896938    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-4sd8r" event={"ID":"a0ea8679-d95b-49d6-a614-9bb4eb4ae029","Type":"ContainerStarted","Data":"a0c325a14805940b90094cb855b54c98c8ada05e195ff4f2aaba41c7f8c00a05"}
Oct 30 05:31:25.897064 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.896975    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-4sd8r" event={"ID":"a0ea8679-d95b-49d6-a614-9bb4eb4ae029","Type":"ContainerStarted","Data":"ca318e520dc7f05fc01002867cc54d58bc9e6382ea5e0c6ada15c4861c8f8ee0"}
Oct 30 05:31:25.899288 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.899259    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2" event={"ID":"41edacc9-dea1-4343-96f5-5caf8a82fe84","Type":"ContainerStarted","Data":"f367b6ce0cb23d3c0cb99db8f880c07c8359f3bc27fd593149ba7f2d801f3ca4"}
Oct 30 05:31:25.900860 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.900834    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb" event={"ID":"0cf86020-761f-4fbc-808e-7dc6c5d5515f","Type":"ContainerStarted","Data":"50eb74303b4645b568e99c96ad8fbeaa01b9f8cc49e117a8b391ccf8104207ad"}
Oct 30 05:31:25.905936 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.905911    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-gh627" event={"ID":"fcdb9d1a-474b-49f3-8683-f3874ad9d769","Type":"ContainerStarted","Data":"0cf66e1d5b0dda93105b97e12d5ab6186e7e81fa3ce2ba18917264ac9bae1648"}
Oct 30 05:31:25.906024 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.905954    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-gh627" event={"ID":"fcdb9d1a-474b-49f3-8683-f3874ad9d769","Type":"ContainerStarted","Data":"2ebca268a5ecc2180eb5e0c3d08af50a88603c4550cf29243ada2b9f338f6567"}
Oct 30 05:31:25.912949 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.912035    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mfzzm" event={"ID":"a655cfec-4ee3-4ac9-badc-4fb318603506","Type":"ContainerStarted","Data":"8e306019d1f5901620e7be039d572cd059f104094e542380c8ba3f76b2bd0133"}
Oct 30 05:31:25.916316 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.916254    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm" event={"ID":"ab824d2b-de9a-4384-a4a8-166f58808be8","Type":"ContainerStarted","Data":"f3d5ba1a086c66afec5f3dfe4bbbc0781dabebdf88b9126df4a0dff0a7794d8a"}
Oct 30 05:31:25.927348 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.927319    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf" event={"ID":"dfbf26d6-b6c8-4d33-9219-5e9fff25a103","Type":"ContainerStarted","Data":"715a9a98d3babaded9b1ff31b402ad532a0dac86c53a91a4d5de0159a9a505f0"}
Oct 30 05:31:25.934355 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.933501    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-k5d59" podStartSLOduration=404.933489645 podStartE2EDuration="6m44.933489645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:24:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:25.933151747 +0000 UTC m=+30.871526127" watchObservedRunningTime="2026-10-30 05:31:25.933489645 +0000 UTC m=+30.871864024"
Oct 30 05:31:25.977795 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:25.977278    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-4sd8r" podStartSLOduration=404.977258898 podStartE2EDuration="6m44.977258898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:24:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:25.976099068 +0000 UTC m=+30.914473451" watchObservedRunningTime="2026-10-30 05:31:25.977258898 +0000 UTC m=+30.915633278"
Oct 30 05:31:26.017466 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:26.017255    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-gh627" podStartSLOduration=405.017236427 podStartE2EDuration="6m45.017236427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:24:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:26.015831326 +0000 UTC m=+30.954205706" watchObservedRunningTime="2026-10-30 05:31:26.017236427 +0000 UTC m=+30.955610813"
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:26.170623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:26.170674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:26.170724    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:26.170761    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:26.170799    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:26.170841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:26.170871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:26.170948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171080    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171139    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls podName:c46db00b-3c48-4d8d-844c-918d8126cc41 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:28.171122212 +0000 UTC m=+33.109496589 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls") pod "dns-default-bzbfz" (UID: "c46db00b-3c48-4d8d-844c-918d8126cc41") : secret "dns-default-metrics-tls" not found
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171532    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle podName:4684e0e6-736f-41bf-8f22-77245e339d58 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:28.171514784 +0000 UTC m=+33.109889160 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle") pod "router-default-59857694c5-wq9fj" (UID: "4684e0e6-736f-41bf-8f22-77245e339d58") : configmap references non-existent config key: service-ca.crt
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171621    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171662    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs podName:4684e0e6-736f-41bf-8f22-77245e339d58 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:28.171647912 +0000 UTC m=+33.110022283 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs") pod "router-default-59857694c5-wq9fj" (UID: "4684e0e6-736f-41bf-8f22-77245e339d58") : secret "router-metrics-certs-default" not found
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171712    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 05:31:26.172160 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171741    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert podName:b0bdcd7b-f471-405a-87b2-ebedb8a1c950 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:28.171731775 +0000 UTC m=+33.110106146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v6b7k" (UID: "b0bdcd7b-f471-405a-87b2-ebedb8a1c950") : secret "networking-console-plugin-cert" not found
Oct 30 05:31:26.173034 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171795    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 05:31:26.173034 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171806    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b668646-vshtx: secret "image-registry-tls" not found
Oct 30 05:31:26.173034 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171837    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls podName:dac2ba5d-8990-47e3-aa65-02ff7500ac01 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:28.171826131 +0000 UTC m=+33.110200502 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls") pod "image-registry-6b668646-vshtx" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01") : secret "image-registry-tls" not found
Oct 30 05:31:26.173034 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171889    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 05:31:26.173034 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171922    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls podName:bfa3e142-50e3-4b85-83fa-46f7dc8b7858 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:28.171911959 +0000 UTC m=+33.110286316 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6xnts" (UID: "bfa3e142-50e3-4b85-83fa-46f7dc8b7858") : secret "cluster-monitoring-operator-tls" not found
Oct 30 05:31:26.173034 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171970    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 05:31:26.173034 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.171997    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls podName:46c256fd-a89b-41f9-95aa-d8e81f783820 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:28.171988623 +0000 UTC m=+33.110362999 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-qsjvb" (UID: "46c256fd-a89b-41f9-95aa-d8e81f783820") : secret "samples-operator-tls" not found
Oct 30 05:31:26.173034 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.172052    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 05:31:26.173034 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.172062    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d65554659-bdvbl: secret "image-registry-tls" not found
Oct 30 05:31:26.173034 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:26.172090    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls podName:e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a nodeName:}" failed. No retries permitted until 2026-10-30 05:31:28.172079588 +0000 UTC m=+33.110453959 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls") pod "image-registry-7d65554659-bdvbl" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a") : secret "image-registry-tls" not found
Oct 30 05:31:26.354661 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:26.354624    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:27.938413 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:27.937664    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq" event={"ID":"2f92fa2c-9133-484f-b326-a12b2b618900","Type":"ContainerStarted","Data":"10bf793074eb998fe2cc81731b530f61f0bf2f762e0e9e090e5bb994bc796757"}
Oct 30 05:31:28.006343 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:28.005249    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq" podStartSLOduration=376.090075214 podStartE2EDuration="6m18.005224671s" totalImagesPullingTime="1.915149457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:27.978466429 +0000 UTC m=+32.916840823" watchObservedRunningTime="2026-10-30 05:31:28.005224671 +0000 UTC m=+32.943599050"
Oct 30 05:31:28.139030 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:28.138988    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:28.203690 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:28.203612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:28.203690 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:28.203660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:28.203916 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:28.203702    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:28.203916 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.203722    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 05:31:28.203916 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.203746    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b668646-vshtx: secret "image-registry-tls" not found
Oct 30 05:31:28.203916 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.203810    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls podName:dac2ba5d-8990-47e3-aa65-02ff7500ac01 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:32.20378678 +0000 UTC m=+37.142161160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls") pod "image-registry-6b668646-vshtx" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01") : secret "image-registry-tls" not found
Oct 30 05:31:28.203916 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.203812    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 05:31:28.203916 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.203822    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d65554659-bdvbl: secret "image-registry-tls" not found
Oct 30 05:31:28.203916 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.203860    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls podName:e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a nodeName:}" failed. No retries permitted until 2026-10-30 05:31:32.203849549 +0000 UTC m=+37.142223906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls") pod "image-registry-7d65554659-bdvbl" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a") : secret "image-registry-tls" not found
Oct 30 05:31:28.203916 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.203905    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 05:31:28.204319 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.203937    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls podName:bfa3e142-50e3-4b85-83fa-46f7dc8b7858 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:32.203927231 +0000 UTC m=+37.142301589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6xnts" (UID: "bfa3e142-50e3-4b85-83fa-46f7dc8b7858") : secret "cluster-monitoring-operator-tls" not found
Oct 30 05:31:28.204319 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.203973    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 05:31:28.204319 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.203992    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls podName:46c256fd-a89b-41f9-95aa-d8e81f783820 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:32.203986262 +0000 UTC m=+37.142360618 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-qsjvb" (UID: "46c256fd-a89b-41f9-95aa-d8e81f783820") : secret "samples-operator-tls" not found
Oct 30 05:31:28.204319 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:28.203732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:28.204319 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:28.204212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:28.204319 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:28.204265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:28.204319 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:28.204290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:28.204900 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:28.204336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:28.204900 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.204346    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 05:31:28.204900 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.204383    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls podName:c46db00b-3c48-4d8d-844c-918d8126cc41 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:32.204371074 +0000 UTC m=+37.142745447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls") pod "dns-default-bzbfz" (UID: "c46db00b-3c48-4d8d-844c-918d8126cc41") : secret "dns-default-metrics-tls" not found
Oct 30 05:31:28.204900 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.204460    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 05:31:28.204900 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.204466    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle podName:4684e0e6-736f-41bf-8f22-77245e339d58 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:32.204454732 +0000 UTC m=+37.142829105 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle") pod "router-default-59857694c5-wq9fj" (UID: "4684e0e6-736f-41bf-8f22-77245e339d58") : configmap references non-existent config key: service-ca.crt
Oct 30 05:31:28.204900 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.204499    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert podName:b0bdcd7b-f471-405a-87b2-ebedb8a1c950 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:32.204486749 +0000 UTC m=+37.142861108 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v6b7k" (UID: "b0bdcd7b-f471-405a-87b2-ebedb8a1c950") : secret "networking-console-plugin-cert" not found
Oct 30 05:31:28.204900 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.204522    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 05:31:28.204900 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:28.204553    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs podName:4684e0e6-736f-41bf-8f22-77245e339d58 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:32.204543259 +0000 UTC m=+37.142917649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs") pod "router-default-59857694c5-wq9fj" (UID: "4684e0e6-736f-41bf-8f22-77245e339d58") : secret "router-metrics-certs-default" not found
Oct 30 05:31:28.300117 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:28.300025    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:28.979517 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:28.979426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:29.317252 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:29.317153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:29.317430 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:29.317282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:29.317562 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:29.317543    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 05:31:29.317653 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:29.317639    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs podName:60bf362c-030f-4adf-9878-842b2fdc326b nodeName:}" failed. No retries permitted until 2026-10-30 05:32:01.317617688 +0000 UTC m=+66.255992048 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs") pod "network-metrics-daemon-pfdj6" (UID: "60bf362c-030f-4adf-9878-842b2fdc326b") : secret "metrics-daemon-secret" not found
Oct 30 05:31:29.322289 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:29.322257    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:29.370224 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:29.370183    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:29.418367 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:29.418327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wkpm\" (UniqueName: \"kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm\") pod \"network-check-target-fdfg7\" (UID: \"f82afdb2-9cea-479b-8d0b-88a814886c3a\") " pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:29.424461 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:29.424431    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wkpm\" (UniqueName: \"kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm\") pod \"network-check-target-fdfg7\" (UID: \"f82afdb2-9cea-479b-8d0b-88a814886c3a\") " pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:29.659839 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:29.659808    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:30.251744 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.251582    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kkcc6"]
Oct 30 05:31:30.266104 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.266083    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fdfg7"]
Oct 30 05:31:30.349128 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:30.349069    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b51a523_ec2a_45ec_b7a1_d0f1d0f778fa.slice/crio-7aab8ee591997264797d019f8c032698f8de028a7bc62a3c490f0f7b524f8afa WatchSource:0}: Error finding container 7aab8ee591997264797d019f8c032698f8de028a7bc62a3c490f0f7b524f8afa: Status 404 returned error can't find the container with id 7aab8ee591997264797d019f8c032698f8de028a7bc62a3c490f0f7b524f8afa
Oct 30 05:31:30.349775 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:30.349752    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf82afdb2_9cea_479b_8d0b_88a814886c3a.slice/crio-31f8ce6b77279e97269bfa4540ec82a0dc92aa008f105d03ca961012c5ba0256 WatchSource:0}: Error finding container 31f8ce6b77279e97269bfa4540ec82a0dc92aa008f105d03ca961012c5ba0256: Status 404 returned error can't find the container with id 31f8ce6b77279e97269bfa4540ec82a0dc92aa008f105d03ca961012c5ba0256
Oct 30 05:31:30.735924 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.735894    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6864g"]
Oct 30 05:31:30.756054 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.756022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.760531 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.760507    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6864g"]
Oct 30 05:31:30.815500 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.815478    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 05:31:30.816941 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.816919    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-4pjwt-q72n8"]
Oct 30 05:31:30.820168 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.820151    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 05:31:30.820259 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.820242    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bmrwn\""
Oct 30 05:31:30.831668 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.831643    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-data-volume\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.831815 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.831694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvw56\" (UniqueName: \"kubernetes.io/projected/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-api-access-cvw56\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.831815 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.831761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.832019 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.831992    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-crio-socket\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.832071 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.832055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.837165 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.837140    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-4pjwt-q72n8"]
Oct 30 05:31:30.837266 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.837245    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:30.852880 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.852855    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lv7bs\""
Oct 30 05:31:30.893150 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.893120    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:30.933208 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.933170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-service-ca-bundle\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:30.933208 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.933207    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-trusted-ca-bundle\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:30.933449 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.933242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.933449 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:30.933308    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 05:31:30.933449 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:30.933387    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm podName:ecfec034-dabf-45fb-a6cb-ef3ace810cb2 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:31.433364062 +0000 UTC m=+36.371738433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6864g" (UID: "ecfec034-dabf-45fb-a6cb-ef3ace810cb2") : configmap "kube-rbac-proxy" not found
Oct 30 05:31:30.933649 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.933502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-crio-socket\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.933649 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.933547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.933737 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.933718    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-crio-socket\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.933771 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:30.933749    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:30.933804 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.933770    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-data-volume\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.933844 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:30.933804    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls podName:ecfec034-dabf-45fb-a6cb-ef3ace810cb2 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:31.433786761 +0000 UTC m=+36.372161123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6864g" (UID: "ecfec034-dabf-45fb-a6cb-ef3ace810cb2") : secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:30.933930 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.933899    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfxnt\" (UniqueName: \"kubernetes.io/projected/f74041a6-7409-439c-9ecb-4bc8dcc2006e-kube-api-access-rfxnt\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:30.934055 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.933978    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/f74041a6-7409-439c-9ecb-4bc8dcc2006e-archives-path\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:30.934055 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.934018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvw56\" (UniqueName: \"kubernetes.io/projected/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-api-access-cvw56\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.934165 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.934055    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-data-volume\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.953573 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.953064    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mfzzm" event={"ID":"a655cfec-4ee3-4ac9-badc-4fb318603506","Type":"ContainerStarted","Data":"8b0f02c8b582b80df7173892cdd1a38cf0bff9762f740a332d7aa6a56d1ebbcd"}
Oct 30 05:31:30.955503 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.955480    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/0.log"
Oct 30 05:31:30.955652 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.955519    2580 generic.go:356] "Generic (PLEG): container finished" podID="ab824d2b-de9a-4384-a4a8-166f58808be8" containerID="0d83f8711a2f58221ab8dfcbd6e1d138525dabad82b551ae26687bcbc430b529" exitCode=255
Oct 30 05:31:30.955652 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.955624    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm" event={"ID":"ab824d2b-de9a-4384-a4a8-166f58808be8","Type":"ContainerDied","Data":"0d83f8711a2f58221ab8dfcbd6e1d138525dabad82b551ae26687bcbc430b529"}
Oct 30 05:31:30.955940 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.955921    2580 scope.go:122] "RemoveContainer" containerID="0d83f8711a2f58221ab8dfcbd6e1d138525dabad82b551ae26687bcbc430b529"
Oct 30 05:31:30.958353 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.958327    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf" event={"ID":"dfbf26d6-b6c8-4d33-9219-5e9fff25a103","Type":"ContainerStarted","Data":"c9d10c82a36b4e76fe20b768b4d8799e7111f00040bfddadfd99f9c9bd0a04af"}
Oct 30 05:31:30.961631 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.961580    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-swtbv" event={"ID":"86fadfbe-f000-4b42-a8ff-7b67061959c5","Type":"ContainerStarted","Data":"99e07dbd2d5e196e824a4da9856bb03b5e06e0dc1a5ca4ac3d9f6adef617b69d"}
Oct 30 05:31:30.964626 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.963375    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvw56\" (UniqueName: \"kubernetes.io/projected/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-api-access-cvw56\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:30.975134 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.975110    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fdfg7" event={"ID":"f82afdb2-9cea-479b-8d0b-88a814886c3a","Type":"ContainerStarted","Data":"31f8ce6b77279e97269bfa4540ec82a0dc92aa008f105d03ca961012c5ba0256"}
Oct 30 05:31:30.978348 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.978260    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2" event={"ID":"41edacc9-dea1-4343-96f5-5caf8a82fe84","Type":"ContainerStarted","Data":"c64589ee522efffc4d6b93cc1f4fe9eb739511511a4d51ae4a654bc60fa6a4d6"}
Oct 30 05:31:30.981219 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.981199    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb" event={"ID":"0cf86020-761f-4fbc-808e-7dc6c5d5515f","Type":"ContainerStarted","Data":"7afadecf0987299fc16f626bb600d6a9ff4bf3f7355221acfd2a44fd16472be6"}
Oct 30 05:31:30.983774 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:30.983715    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kkcc6" event={"ID":"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa","Type":"ContainerStarted","Data":"7aab8ee591997264797d019f8c032698f8de028a7bc62a3c490f0f7b524f8afa"}
Oct 30 05:31:31.003572 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.003518    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mfzzm" podStartSLOduration=1.846214656 podStartE2EDuration="7.003502145s" totalImagesPullingTime="5.157287489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:31.0028124 +0000 UTC m=+35.941186806" watchObservedRunningTime="2026-10-30 05:31:31.003502145 +0000 UTC m=+35.941876558"
Oct 30 05:31:31.035024 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.034993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxnt\" (UniqueName: \"kubernetes.io/projected/f74041a6-7409-439c-9ecb-4bc8dcc2006e-kube-api-access-rfxnt\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:31.035122 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.035075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/f74041a6-7409-439c-9ecb-4bc8dcc2006e-archives-path\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:31.035175 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.035119    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-service-ca-bundle\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:31.035175 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.035142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-trusted-ca-bundle\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:31.038068 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.037346    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/f74041a6-7409-439c-9ecb-4bc8dcc2006e-archives-path\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:31.038068 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.037466    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-trusted-ca-bundle\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:31.038068 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.037830    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-service-ca-bundle\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:31.064164 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.064112    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf" podStartSLOduration=369.56742348 podStartE2EDuration="6m15.064092947s" totalImagesPullingTime="5.496669467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:31.043142159 +0000 UTC m=+35.981516537" watchObservedRunningTime="2026-10-30 05:31:31.064092947 +0000 UTC m=+36.002467326"
Oct 30 05:31:31.064684 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.064670    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9"]
Oct 30 05:31:31.068583 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.068474    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:31.069426 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.069400    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfxnt\" (UniqueName: \"kubernetes.io/projected/f74041a6-7409-439c-9ecb-4bc8dcc2006e-kube-api-access-rfxnt\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:31.074750 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.074657    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9"
Oct 30 05:31:31.077994 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.076530    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd"]
Oct 30 05:31:31.092349 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.089672    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb" podStartSLOduration=378.354394874 podStartE2EDuration="6m23.089654075s" totalImagesPullingTime="4.735259201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:31.089112952 +0000 UTC m=+36.027487324" watchObservedRunningTime="2026-10-30 05:31:31.089654075 +0000 UTC m=+36.028028450"
Oct 30 05:31:31.094136 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.094110    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qrqvm\""
Oct 30 05:31:31.094987 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.094961    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9"]
Oct 30 05:31:31.095100 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.095077    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd"
Oct 30 05:31:31.095915 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.095841    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 05:31:31.095915 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.095851    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:31.122307 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.119786    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd"]
Oct 30 05:31:31.136825 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.136793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9hln\" (UniqueName: \"kubernetes.io/projected/834a0562-2972-449c-9ec4-a69908732c37-kube-api-access-z9hln\") pod \"migrator-66df5d496b-rq4pd\" (UID: \"834a0562-2972-449c-9ec4-a69908732c37\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd"
Oct 30 05:31:31.136993 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.136964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwdkg\" (UniqueName: \"kubernetes.io/projected/abe28d56-457c-4e87-9cb1-804247d48da0-kube-api-access-dwdkg\") pod \"migrator-66df5d496b-n6zg9\" (UID: \"abe28d56-457c-4e87-9cb1-804247d48da0\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9"
Oct 30 05:31:31.138001 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.137958    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2" podStartSLOduration=378.2939055 podStartE2EDuration="6m23.137946069s" totalImagesPullingTime="4.844040569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:31.137509586 +0000 UTC m=+36.075883966" watchObservedRunningTime="2026-10-30 05:31:31.137946069 +0000 UTC m=+36.076320448"
Oct 30 05:31:31.164462 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.164431    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:31.182988 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.182938    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-swtbv" podStartSLOduration=376.439901768 podStartE2EDuration="6m21.182921134s" totalImagesPullingTime="4.743019366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:31.182216841 +0000 UTC m=+36.120591221" watchObservedRunningTime="2026-10-30 05:31:31.182921134 +0000 UTC m=+36.121295512"
Oct 30 05:31:31.238936 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.238860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwdkg\" (UniqueName: \"kubernetes.io/projected/abe28d56-457c-4e87-9cb1-804247d48da0-kube-api-access-dwdkg\") pod \"migrator-66df5d496b-n6zg9\" (UID: \"abe28d56-457c-4e87-9cb1-804247d48da0\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9"
Oct 30 05:31:31.239087 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.238978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9hln\" (UniqueName: \"kubernetes.io/projected/834a0562-2972-449c-9ec4-a69908732c37-kube-api-access-z9hln\") pod \"migrator-66df5d496b-rq4pd\" (UID: \"834a0562-2972-449c-9ec4-a69908732c37\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd"
Oct 30 05:31:31.260020 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.258566    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:31.262789 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.262742    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwdkg\" (UniqueName: \"kubernetes.io/projected/abe28d56-457c-4e87-9cb1-804247d48da0-kube-api-access-dwdkg\") pod \"migrator-66df5d496b-n6zg9\" (UID: \"abe28d56-457c-4e87-9cb1-804247d48da0\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9"
Oct 30 05:31:31.262993 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.262975    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9hln\" (UniqueName: \"kubernetes.io/projected/834a0562-2972-449c-9ec4-a69908732c37-kube-api-access-z9hln\") pod \"migrator-66df5d496b-rq4pd\" (UID: \"834a0562-2972-449c-9ec4-a69908732c37\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd"
Oct 30 05:31:31.298178 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.298143    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-6xwxq"]
Oct 30 05:31:31.328270 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.328236    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-6xwxq"]
Oct 30 05:31:31.328483 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.328343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:31.339645 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.339616    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/06639c1a-71fb-47d6-898f-464546d8b66a-signing-key\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:31.339791 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.339659    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/06639c1a-71fb-47d6-898f-464546d8b66a-signing-cabundle\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:31.339791 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.339724    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l9nz\" (UniqueName: \"kubernetes.io/projected/06639c1a-71fb-47d6-898f-464546d8b66a-kube-api-access-9l9nz\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:31.343219 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.343195    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 05:31:31.343355 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.343231    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 05:31:31.343355 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.343301    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 05:31:31.343455 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.343392    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 05:31:31.343507 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.343458    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-st5hg\""
Oct 30 05:31:31.417362 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.417318    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:31.432301 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.432277    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9"
Oct 30 05:31:31.441080 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.441050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:31.441224 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.441111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/06639c1a-71fb-47d6-898f-464546d8b66a-signing-key\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:31.441224 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.441140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/06639c1a-71fb-47d6-898f-464546d8b66a-signing-cabundle\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:31.441224 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.441198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l9nz\" (UniqueName: \"kubernetes.io/projected/06639c1a-71fb-47d6-898f-464546d8b66a-kube-api-access-9l9nz\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:31.441375 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:31.441220    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:31.441375 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.441237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:31.441375 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:31.441290    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls podName:ecfec034-dabf-45fb-a6cb-ef3ace810cb2 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:32.441270343 +0000 UTC m=+37.379644704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6864g" (UID: "ecfec034-dabf-45fb-a6cb-ef3ace810cb2") : secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:31.441375 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:31.441317    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 05:31:31.441695 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:31.441676    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm podName:ecfec034-dabf-45fb-a6cb-ef3ace810cb2 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:32.441659741 +0000 UTC m=+37.380034114 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6864g" (UID: "ecfec034-dabf-45fb-a6cb-ef3ace810cb2") : configmap "kube-rbac-proxy" not found
Oct 30 05:31:31.441961 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.441936    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/06639c1a-71fb-47d6-898f-464546d8b66a-signing-cabundle\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:31.444656 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.444637    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/06639c1a-71fb-47d6-898f-464546d8b66a-signing-key\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:31.453420 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.453398    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd"
Oct 30 05:31:31.465231 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.465197    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l9nz\" (UniqueName: \"kubernetes.io/projected/06639c1a-71fb-47d6-898f-464546d8b66a-kube-api-access-9l9nz\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:31.493622 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.492895    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-4pjwt-q72n8"]
Oct 30 05:31:31.497741 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:31.497710    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf74041a6_7409_439c_9ecb_4bc8dcc2006e.slice/crio-a5112a1b37f0d6d31f1c8c74496d6cc647a08bf2457a218f0b932b903ddd4728 WatchSource:0}: Error finding container a5112a1b37f0d6d31f1c8c74496d6cc647a08bf2457a218f0b932b903ddd4728: Status 404 returned error can't find the container with id a5112a1b37f0d6d31f1c8c74496d6cc647a08bf2457a218f0b932b903ddd4728
Oct 30 05:31:31.668206 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.668166    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:31.718351 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.718192    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:31.765194 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.765166    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9"]
Oct 30 05:31:31.767352 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:31.767308    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe28d56_457c_4e87_9cb1_804247d48da0.slice/crio-82e9d2dc6491dce1a0f7058d9ce6749095a22608155a988484a0c045de971af0 WatchSource:0}: Error finding container 82e9d2dc6491dce1a0f7058d9ce6749095a22608155a988484a0c045de971af0: Status 404 returned error can't find the container with id 82e9d2dc6491dce1a0f7058d9ce6749095a22608155a988484a0c045de971af0
Oct 30 05:31:31.813093 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.813061    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd"]
Oct 30 05:31:31.816252 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:31.816220    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834a0562_2972_449c_9ec4_a69908732c37.slice/crio-1d626d33ff578d8387a5281f5d535b1fc8664015abaff1af922173eb580defd8 WatchSource:0}: Error finding container 1d626d33ff578d8387a5281f5d535b1fc8664015abaff1af922173eb580defd8: Status 404 returned error can't find the container with id 1d626d33ff578d8387a5281f5d535b1fc8664015abaff1af922173eb580defd8
Oct 30 05:31:31.901913 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.901821    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:31.995919 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.995842    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-6xwxq"]
Oct 30 05:31:31.996388 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.996367    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:31:31.996897 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.996877    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/0.log"
Oct 30 05:31:31.996989 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.996923    2580 generic.go:356] "Generic (PLEG): container finished" podID="ab824d2b-de9a-4384-a4a8-166f58808be8" containerID="b6927366bfdc148db909b2ffc636d98277afcb80bc1db53d386c12c71c939fb8" exitCode=255
Oct 30 05:31:31.997056 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.997034    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm" event={"ID":"ab824d2b-de9a-4384-a4a8-166f58808be8","Type":"ContainerDied","Data":"b6927366bfdc148db909b2ffc636d98277afcb80bc1db53d386c12c71c939fb8"}
Oct 30 05:31:31.997121 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.997112    2580 scope.go:122] "RemoveContainer" containerID="0d83f8711a2f58221ab8dfcbd6e1d138525dabad82b551ae26687bcbc430b529"
Oct 30 05:31:31.997292 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.997268    2580 scope.go:122] "RemoveContainer" containerID="b6927366bfdc148db909b2ffc636d98277afcb80bc1db53d386c12c71c939fb8"
Oct 30 05:31:31.997511 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:31.997487    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-58dpm_openshift-console-operator(ab824d2b-de9a-4384-a4a8-166f58808be8)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm" podUID="ab824d2b-de9a-4384-a4a8-166f58808be8"
Oct 30 05:31:31.998387 ip-10-0-136-173 kubenswrapper[2580]: W1030 05:31:31.998357    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06639c1a_71fb_47d6_898f_464546d8b66a.slice/crio-9c2b138d5586b27e31708785a0909a45b2a2168eb682c7d19a46c73a9b094d64 WatchSource:0}: Error finding container 9c2b138d5586b27e31708785a0909a45b2a2168eb682c7d19a46c73a9b094d64: Status 404 returned error can't find the container with id 9c2b138d5586b27e31708785a0909a45b2a2168eb682c7d19a46c73a9b094d64
Oct 30 05:31:31.999570 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.999546    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fdfg7" event={"ID":"f82afdb2-9cea-479b-8d0b-88a814886c3a","Type":"ContainerStarted","Data":"c15223540039fb258c9e35485adb716dba5462196c6796c08a25d279cd831184"}
Oct 30 05:31:31.999676 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:31.999663    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:32.000911 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.000891    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9" event={"ID":"abe28d56-457c-4e87-9cb1-804247d48da0","Type":"ContainerStarted","Data":"82e9d2dc6491dce1a0f7058d9ce6749095a22608155a988484a0c045de971af0"}
Oct 30 05:31:32.002705 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.002677    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-4pjwt-q72n8" event={"ID":"f74041a6-7409-439c-9ecb-4bc8dcc2006e","Type":"ContainerStarted","Data":"ac3eacc39e302d2292f01fd39af46ebdff774d7f1cec718e23e931c257f108da"}
Oct 30 05:31:32.002786 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.002728    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-4pjwt-q72n8" event={"ID":"f74041a6-7409-439c-9ecb-4bc8dcc2006e","Type":"ContainerStarted","Data":"a5112a1b37f0d6d31f1c8c74496d6cc647a08bf2457a218f0b932b903ddd4728"}
Oct 30 05:31:32.004947 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.004923    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd" event={"ID":"834a0562-2972-449c-9ec4-a69908732c37","Type":"ContainerStarted","Data":"1d626d33ff578d8387a5281f5d535b1fc8664015abaff1af922173eb580defd8"}
Oct 30 05:31:32.086744 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.086686    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fdfg7" podStartSLOduration=35.519154078 podStartE2EDuration="36.086665575s" totalImagesPullingTime="567.511497ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:32.085041679 +0000 UTC m=+37.023416059" watchObservedRunningTime="2026-10-30 05:31:32.086665575 +0000 UTC m=+37.025039974"
Oct 30 05:31:32.125029 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.124963    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-4pjwt-q72n8" podStartSLOduration=2.124943516 podStartE2EDuration="2.124943516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:31:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:32.12398389 +0000 UTC m=+37.062358270" watchObservedRunningTime="2026-10-30 05:31:32.124943516 +0000 UTC m=+37.063317897"
Oct 30 05:31:32.160751 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.160713    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:32.250290 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.250252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:32.250456 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.250305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:32.250456 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.250324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:32.250456 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.250352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:32.250456 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.250385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:32.250456 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.250418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:32.250456 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.250447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:32.250803 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.250479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:32.250803 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.250641    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 05:31:32.250803 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.250656    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d65554659-bdvbl: secret "image-registry-tls" not found
Oct 30 05:31:32.250803 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.250717    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls podName:e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a nodeName:}" failed. No retries permitted until 2026-10-30 05:31:40.250703447 +0000 UTC m=+45.189077804 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls") pod "image-registry-7d65554659-bdvbl" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a") : secret "image-registry-tls" not found
Oct 30 05:31:32.251043 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251014    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 05:31:32.251180 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251061    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls podName:c46db00b-3c48-4d8d-844c-918d8126cc41 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:40.251048687 +0000 UTC m=+45.189423044 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls") pod "dns-default-bzbfz" (UID: "c46db00b-3c48-4d8d-844c-918d8126cc41") : secret "dns-default-metrics-tls" not found
Oct 30 05:31:32.251180 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251123    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle podName:4684e0e6-736f-41bf-8f22-77245e339d58 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:40.251113789 +0000 UTC m=+45.189488145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle") pod "router-default-59857694c5-wq9fj" (UID: "4684e0e6-736f-41bf-8f22-77245e339d58") : configmap references non-existent config key: service-ca.crt
Oct 30 05:31:32.251310 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251183    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 05:31:32.251310 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251214    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs podName:4684e0e6-736f-41bf-8f22-77245e339d58 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:40.251204684 +0000 UTC m=+45.189579042 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs") pod "router-default-59857694c5-wq9fj" (UID: "4684e0e6-736f-41bf-8f22-77245e339d58") : secret "router-metrics-certs-default" not found
Oct 30 05:31:32.251310 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251265    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 05:31:32.251310 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251298    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert podName:b0bdcd7b-f471-405a-87b2-ebedb8a1c950 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:40.251285992 +0000 UTC m=+45.189660351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-v6b7k" (UID: "b0bdcd7b-f471-405a-87b2-ebedb8a1c950") : secret "networking-console-plugin-cert" not found
Oct 30 05:31:32.251519 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251358    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 05:31:32.251519 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251369    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b668646-vshtx: secret "image-registry-tls" not found
Oct 30 05:31:32.251519 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251398    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls podName:dac2ba5d-8990-47e3-aa65-02ff7500ac01 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:40.251388313 +0000 UTC m=+45.189762695 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls") pod "image-registry-6b668646-vshtx" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01") : secret "image-registry-tls" not found
Oct 30 05:31:32.251519 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251467    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 05:31:32.251519 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251497    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls podName:bfa3e142-50e3-4b85-83fa-46f7dc8b7858 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:40.251487544 +0000 UTC m=+45.189861903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6xnts" (UID: "bfa3e142-50e3-4b85-83fa-46f7dc8b7858") : secret "cluster-monitoring-operator-tls" not found
Oct 30 05:31:32.251812 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251550    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 05:31:32.251812 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.251579    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls podName:46c256fd-a89b-41f9-95aa-d8e81f783820 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:40.251570195 +0000 UTC m=+45.189944559 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-qsjvb" (UID: "46c256fd-a89b-41f9-95aa-d8e81f783820") : secret "samples-operator-tls" not found
Oct 30 05:31:32.350888 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.350801    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:32.452365 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.452333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:32.452532 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.452448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:32.452663 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.452628    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 05:31:32.452742 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.452700    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm podName:ecfec034-dabf-45fb-a6cb-ef3ace810cb2 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:34.45267941 +0000 UTC m=+39.391053787 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6864g" (UID: "ecfec034-dabf-45fb-a6cb-ef3ace810cb2") : configmap "kube-rbac-proxy" not found
Oct 30 05:31:32.453090 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.453069    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:32.453234 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:32.453131    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls podName:ecfec034-dabf-45fb-a6cb-ef3ace810cb2 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:34.453116508 +0000 UTC m=+39.391490879 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6864g" (UID: "ecfec034-dabf-45fb-a6cb-ef3ace810cb2") : secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:32.521079 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:32.521022    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:33.014357 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:33.014319    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq" event={"ID":"06639c1a-71fb-47d6-898f-464546d8b66a","Type":"ContainerStarted","Data":"b7180dd412345d40e0b02bd1a83868299571d214e7753ebe0c7518422ecd9f74"}
Oct 30 05:31:33.014562 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:33.014372    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq" event={"ID":"06639c1a-71fb-47d6-898f-464546d8b66a","Type":"ContainerStarted","Data":"9c2b138d5586b27e31708785a0909a45b2a2168eb682c7d19a46c73a9b094d64"}
Oct 30 05:31:33.017926 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:33.017255    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:31:33.018486 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:33.018464    2580 scope.go:122] "RemoveContainer" containerID="b6927366bfdc148db909b2ffc636d98277afcb80bc1db53d386c12c71c939fb8"
Oct 30 05:31:33.018781 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:33.018720    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-58dpm_openshift-console-operator(ab824d2b-de9a-4384-a4a8-166f58808be8)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm" podUID="ab824d2b-de9a-4384-a4a8-166f58808be8"
Oct 30 05:31:33.111293 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:33.111226    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq" podStartSLOduration=2.111207032 podStartE2EDuration="2.111207032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:31:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:33.062748958 +0000 UTC m=+38.001123336" watchObservedRunningTime="2026-10-30 05:31:33.111207032 +0000 UTC m=+38.049581413"
Oct 30 05:31:33.234125 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:33.234017    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:33.400648 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:33.395517    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:33.564745 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:33.564710    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:33.738973 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:33.738932    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:33.987390 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:33.987311    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mtj5z_b4c2b487-ec63-4b07-a11c-781cb9afd37d/dns-node-resolver/0.log"
Oct 30 05:31:34.023366 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.023329    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd" event={"ID":"834a0562-2972-449c-9ec4-a69908732c37","Type":"ContainerStarted","Data":"3b162c2499bbb969113594fa7e4310e05d67db91f6253f8ee78a6d2ceced90bc"}
Oct 30 05:31:34.025025 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.024997    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9" event={"ID":"abe28d56-457c-4e87-9cb1-804247d48da0","Type":"ContainerStarted","Data":"879ad78363a0642c26725184e3e9d4310c33d886dbc7affe8514c5cb5bcd2454"}
Oct 30 05:31:34.064802 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.064747    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd" podStartSLOduration=1.256543711 podStartE2EDuration="3.064727541s" totalImagesPullingTime="1.80818383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:31:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:34.064002682 +0000 UTC m=+39.002377060" watchObservedRunningTime="2026-10-30 05:31:34.064727541 +0000 UTC m=+39.003101919"
Oct 30 05:31:34.108369 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.108299    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9" podStartSLOduration=1.256020011 podStartE2EDuration="3.10828122s" totalImagesPullingTime="1.852261209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:31:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:34.103011014 +0000 UTC m=+39.041385394" watchObservedRunningTime="2026-10-30 05:31:34.10828122 +0000 UTC m=+39.046655598"
Oct 30 05:31:34.385045 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.384934    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:34.476485 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.476446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:34.476979 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.476582    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:34.476979 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:34.476583    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 05:31:34.476979 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:34.476720    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm podName:ecfec034-dabf-45fb-a6cb-ef3ace810cb2 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:38.476704231 +0000 UTC m=+43.415078588 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6864g" (UID: "ecfec034-dabf-45fb-a6cb-ef3ace810cb2") : configmap "kube-rbac-proxy" not found
Oct 30 05:31:34.476979 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:34.476759    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:34.476979 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:34.476829    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls podName:ecfec034-dabf-45fb-a6cb-ef3ace810cb2 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:38.476813994 +0000 UTC m=+43.415188375 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6864g" (UID: "ecfec034-dabf-45fb-a6cb-ef3ace810cb2") : secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:34.551980 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.551930    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:34.774340 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.774212    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:34.826266 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.826225    2580 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:34.826441 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.826317    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:34.826758 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.826741    2580 scope.go:122] "RemoveContainer" containerID="b6927366bfdc148db909b2ffc636d98277afcb80bc1db53d386c12c71c939fb8"
Oct 30 05:31:34.826970 ip-10-0-136-173 kubenswrapper[2580]: E1030 05:31:34.826949    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-58dpm_openshift-console-operator(ab824d2b-de9a-4384-a4a8-166f58808be8)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm" podUID="ab824d2b-de9a-4384-a4a8-166f58808be8"
Oct 30 05:31:34.970486 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:34.970451    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:35.173386 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:35.173346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:35.178420 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:35.178397    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qw6mh_14a30f61-33a2-4f29-ba24-fc02c0c6cc15/node-ca/0.log"
Oct 30 05:31:35.330616 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:35.330553    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:35.530488 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:35.530410    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:35.578788 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:35.578763    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mfzzm_a655cfec-4ee3-4ac9-badc-4fb318603506/serve-healthcheck-canary/0.log"
Oct 30 05:31:35.779582 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:35.779556    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-n6zg9_abe28d56-457c-4e87-9cb1-804247d48da0/migrator/0.log"
Oct 30 05:31:35.928225 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:35.928195    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:35.979470 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:35.979440    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-rq4pd_834a0562-2972-449c-9ec4-a69908732c37/migrator/0.log"
Oct 30 05:31:36.036105 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:36.036070    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kkcc6" event={"ID":"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa","Type":"ContainerStarted","Data":"b30e74a98a6a6ff5969e043f6df7c9bca4913df89f4c1d92150f25bd26f217b6"}
Oct 30 05:31:36.075740 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:36.075687    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kkcc6" podStartSLOduration=34.567353828 podStartE2EDuration="40.075672829s" totalImagesPullingTime="5.508319001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 05:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:36.074800978 +0000 UTC m=+41.013175356" watchObservedRunningTime="2026-10-30 05:31:36.075672829 +0000 UTC m=+41.014047231"
Oct 30 05:31:36.183004 ip-10-0-136-173 kubenswrapper[2580]: I1030 05:31:36.182976    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-j55lb_0cf86020-761f-4fbc-808e-7dc6c5d5515f/kube-storage-version-migrator-operator/0.log"
Oct 30 05:31:36.191913 ip-10-0-136-173 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 05:31:36.203073 ip-10-0-136-173 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 05:31:36.203306 ip-10-0-136-173 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 05:31:36.203521 ip-10-0-136-173 systemd[1]: kubelet.service: Consumed 3.345s CPU time.
Oct 30 05:31:36.222111 ip-10-0-136-173 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 05:31:36.317013 ip-10-0-136-173 kubenswrapper[5066]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 05:31:36.317013 ip-10-0-136-173 kubenswrapper[5066]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 05:31:36.317013 ip-10-0-136-173 kubenswrapper[5066]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 05:31:36.317013 ip-10-0-136-173 kubenswrapper[5066]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 05:31:36.319111 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319025    5066 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:31:36.319111 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319112    5066 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319117    5066 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319122    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319126    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319129    5066 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319132    5066 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319134    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319137    5066 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319140    5066 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319143    5066 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319146    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319148    5066 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319151    5066 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319153    5066 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319156    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319158    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319161    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319164    5066 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319166    5066 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:31:36.319191 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319169    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319180    5066 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319183    5066 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319185    5066 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319188    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319191    5066 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319194    5066 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319197    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319199    5066 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319202    5066 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319205    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319208    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319211    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319213    5066 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319216    5066 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319218    5066 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319221    5066 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319224    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319226    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319230    5066 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:31:36.319677 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319233    5066 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319235    5066 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319238    5066 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319241    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319243    5066 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319246    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319249    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319251    5066 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319254    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319256    5066 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319259    5066 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319261    5066 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319264    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319266    5066 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319270    5066 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319273    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319277    5066 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319281    5066 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319285    5066 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:31:36.320159 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319288    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319291    5066 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319293    5066 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319296    5066 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319299    5066 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319301    5066 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319304    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319307    5066 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319310    5066 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319312    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319315    5066 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319317    5066 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319320    5066 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319323    5066 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319326    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319328    5066 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319331    5066 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319333    5066 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319336    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:31:36.320643 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319338    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319341    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319343    5066 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319345    5066 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319348    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319350    5066 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319353    5066 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319356    5066 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319358    5066 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319361    5066 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319363    5066 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319366    5066 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319368    5066 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319371    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319374    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319376    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319379    5066 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319382    5066 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319385    5066 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319387    5066 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:31:36.321121 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319389    5066 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319392    5066 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319394    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319397    5066 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319399    5066 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319402    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319404    5066 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.319408    5066 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320203    5066 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320210    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320214    5066 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320218    5066 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320221    5066 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320224    5066 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320226    5066 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320229    5066 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320232    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320235    5066 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320238    5066 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320241    5066 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:31:36.321620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320243    5066 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320246    5066 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320249    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320252    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320254    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320257    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320260    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320263    5066 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320265    5066 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320268    5066 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320271    5066 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320273    5066 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320276    5066 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320278    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320281    5066 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320283    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320286    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320288    5066 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320291    5066 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320293    5066 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:31:36.322158 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320297    5066 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320300    5066 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320303    5066 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320305    5066 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320308    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320310    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320313    5066 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320315    5066 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320318    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320320    5066 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320323    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320326    5066 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320328    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320331    5066 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320333    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320336    5066 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320338    5066 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320340    5066 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320343    5066 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320346    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:31:36.322652 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320351    5066 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320354    5066 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320357    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320359    5066 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320362    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320364    5066 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320367    5066 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320369    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320371    5066 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320374    5066 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320377    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320379    5066 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320382    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320387    5066 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320390    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320393    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320396    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320399    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320401    5066 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:31:36.323132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320404    5066 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320406    5066 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320409    5066 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320411    5066 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320414    5066 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320416    5066 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320419    5066 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320421    5066 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320424    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320428    5066 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320432    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320436    5066 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320439    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320443    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320446    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320449    5066 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320452    5066 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320455    5066 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:31:36.323644 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320459    5066 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320462    5066 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320465    5066 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320468    5066 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320471    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320473    5066 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320476    5066 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320478    5066 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320481    5066 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320489    5066 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320492    5066 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320495    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320497    5066 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320499    5066 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320502    5066 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320504    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.320508    5066 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321032    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321038    5066 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321041    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:31:36.324075 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321043    5066 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321046    5066 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321049    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321051    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321054    5066 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321057    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321060    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321064    5066 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321067    5066 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321069    5066 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321072    5066 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321075    5066 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321077    5066 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321080    5066 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321082    5066 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321085    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321087    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321090    5066 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321092    5066 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:31:36.324557 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321095    5066 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321097    5066 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321119    5066 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321122    5066 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321124    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321127    5066 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321130    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321133    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321137    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321140    5066 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321148    5066 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321151    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321154    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321156    5066 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321159    5066 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321162    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321165    5066 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321170    5066 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321174    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:31:36.325022 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321177    5066 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321180    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321183    5066 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321186    5066 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321189    5066 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321192    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321194    5066 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321197    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321200    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321202    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321205    5066 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321207    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321210    5066 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321212    5066 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321214    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321217    5066 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321225    5066 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321229    5066 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321231    5066 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321234    5066 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:31:36.325775 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321236    5066 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321239    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321241    5066 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321244    5066 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321246    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321249    5066 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321251    5066 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321254    5066 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321256    5066 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321259    5066 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321261    5066 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321264    5066 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321266    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321269    5066 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321272    5066 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321274    5066 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321277    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321279    5066 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321282    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:31:36.326631 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321284    5066 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321287    5066 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321290    5066 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321292    5066 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321295    5066 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321298    5066 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321300    5066 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321303    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321305    5066 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321308    5066 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321318    5066 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321321    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321323    5066 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321326    5066 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321328    5066 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321331    5066 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321334    5066 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321336    5066 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321338    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321341    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:31:36.327429 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321343    5066 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321346    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321348    5066 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321351    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321357    5066 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.321360    5066 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321455    5066 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321462    5066 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321475    5066 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321479    5066 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321484    5066 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321487    5066 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321494    5066 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321498    5066 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321501    5066 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321504    5066 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321507    5066 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321510    5066 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321513    5066 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321516    5066 flags.go:64] FLAG: --cgroup-root=""
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321519    5066 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321522    5066 flags.go:64] FLAG: --client-ca-file=""
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321525    5066 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 05:31:36.328101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321534    5066 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321540    5066 flags.go:64] FLAG: --cluster-domain=""
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321543    5066 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321546    5066 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321549    5066 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321553    5066 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321556    5066 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321559    5066 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321562    5066 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321565    5066 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321568    5066 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321571    5066 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321574    5066 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321578    5066 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321581    5066 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321599    5066 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321603    5066 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321605    5066 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321608    5066 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321611    5066 flags.go:64] FLAG: --enable-server="true"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321614    5066 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321623    5066 flags.go:64] FLAG: --event-burst="100"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321627    5066 flags.go:64] FLAG: --event-qps="50"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321630    5066 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 05:31:36.328963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321633    5066 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321636    5066 flags.go:64] FLAG: --eviction-hard=""
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321641    5066 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321644    5066 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321647    5066 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321650    5066 flags.go:64] FLAG: --eviction-soft=""
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321653    5066 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321657    5066 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321660    5066 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321669    5066 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321672    5066 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321675    5066 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321678    5066 flags.go:64] FLAG: --feature-gates=""
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321682    5066 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321685    5066 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321688    5066 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321691    5066 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321694    5066 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321697    5066 flags.go:64] FLAG: --help="false"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321701    5066 flags.go:64] FLAG: --hostname-override="ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321704    5066 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321708    5066 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321711    5066 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321715    5066 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 05:31:36.329699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321718    5066 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321721    5066 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321723    5066 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321726    5066 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321729    5066 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321732    5066 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321735    5066 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321738    5066 flags.go:64] FLAG: --kube-reserved=""
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321741    5066 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321743    5066 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321746    5066 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321749    5066 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321752    5066 flags.go:64] FLAG: --lock-file=""
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321754    5066 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321757    5066 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321761    5066 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321767    5066 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321770    5066 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321779    5066 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321782    5066 flags.go:64] FLAG: --logging-format="text"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321785    5066 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321788    5066 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321791    5066 flags.go:64] FLAG: --manifest-url=""
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321794    5066 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321798    5066 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 05:31:36.330291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321801    5066 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321805    5066 flags.go:64] FLAG: --max-pods="110"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321808    5066 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321811    5066 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321815    5066 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321818    5066 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321820    5066 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321823    5066 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321826    5066 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321833    5066 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321836    5066 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321839    5066 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321842    5066 flags.go:64] FLAG: --pod-cidr=""
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321845    5066 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321847    5066 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321850    5066 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321853    5066 flags.go:64] FLAG: --port="10250"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321856    5066 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321859    5066 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-098a57ab22afcd504"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321862    5066 flags.go:64] FLAG: --qos-reserved=""
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321865    5066 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321868    5066 flags.go:64] FLAG: --register-node="true"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321872    5066 flags.go:64] FLAG: --register-with-taints=""
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321876    5066 flags.go:64] FLAG: --registry-burst="10"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321879    5066 flags.go:64] FLAG: --registry-qps="5"
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321882    5066 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 05:31:36.330921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321891    5066 flags.go:64] FLAG: --reserved-memory=""
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321895    5066 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321898    5066 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321901    5066 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321904    5066 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321906    5066 flags.go:64] FLAG: --runonce="false"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321909    5066 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321912    5066 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321915    5066 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321918    5066 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321922    5066 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321925    5066 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321928    5066 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321931    5066 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321934    5066 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321937    5066 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321940    5066 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321943    5066 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321946    5066 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321949    5066 flags.go:64] FLAG: --system-cgroups=""
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321951    5066 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321956    5066 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321959    5066 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321962    5066 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321968    5066 flags.go:64] FLAG: --tls-min-version=""
Oct 30 05:31:36.331526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321971    5066 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321974    5066 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321977    5066 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321981    5066 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321984    5066 flags.go:64] FLAG: --v="2"
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321988    5066 flags.go:64] FLAG: --version="false"
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321992    5066 flags.go:64] FLAG: --vmodule=""
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.321996    5066 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.322005    5066 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322186    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322190    5066 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322193    5066 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322196    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322198    5066 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322201    5066 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322203    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322206    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322210    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322212    5066 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322215    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322218    5066 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322220    5066 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:31:36.332194 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322223    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322226    5066 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322228    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322230    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322233    5066 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322235    5066 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322238    5066 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322241    5066 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322243    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322246    5066 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322248    5066 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322250    5066 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322253    5066 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322257    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322259    5066 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322262    5066 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322264    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322267    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322269    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322279    5066 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:31:36.332751 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322282    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322284    5066 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322287    5066 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322289    5066 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322292    5066 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322294    5066 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322297    5066 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322299    5066 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322302    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322304    5066 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322307    5066 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322309    5066 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322312    5066 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322314    5066 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322317    5066 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322319    5066 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322322    5066 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322324    5066 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322327    5066 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322329    5066 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:31:36.333249 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322332    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322334    5066 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322337    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322339    5066 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322342    5066 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322346    5066 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322348    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322351    5066 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322353    5066 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322356    5066 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322358    5066 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322360    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322369    5066 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322371    5066 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322374    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322376    5066 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322380    5066 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322384    5066 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322387    5066 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:31:36.333747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322390    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322392    5066 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322395    5066 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322397    5066 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322400    5066 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322402    5066 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322405    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322407    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322410    5066 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322412    5066 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322420    5066 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322423    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322426    5066 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322428    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322431    5066 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322434    5066 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322436    5066 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322438    5066 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322442    5066 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322445    5066 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:31:36.334262 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322447    5066 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322450    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322452    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322455    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322457    5066 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322459    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322472    5066 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322475    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322478    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322482    5066 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322485    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322488    5066 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322491    5066 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322494    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.322521    5066 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 05:31:36.334926 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322569    5066 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322573    5066 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322577    5066 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322579    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322582    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322597    5066 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322600    5066 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322603    5066 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322606    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322608    5066 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322611    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322614    5066 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322616    5066 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322619    5066 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322621    5066 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322625    5066 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322631    5066 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322634    5066 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322637    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322639    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:31:36.335298 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322642    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322644    5066 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322647    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322649    5066 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322659    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322662    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322666    5066 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322671    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322675    5066 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322678    5066 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322681    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322683    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322686    5066 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322689    5066 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322693    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322696    5066 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322698    5066 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322701    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322704    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:31:36.335833 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322713    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322716    5066 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322719    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322721    5066 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322724    5066 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322726    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322729    5066 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322731    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322735    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322737    5066 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322740    5066 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322742    5066 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322745    5066 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322748    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322750    5066 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322753    5066 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322755    5066 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322758    5066 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322767    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322769    5066 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:31:36.336306 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322772    5066 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322774    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322777    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322779    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322782    5066 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322784    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322787    5066 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322791    5066 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322793    5066 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322809    5066 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322813    5066 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322816    5066 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322819    5066 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322823    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322826    5066 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322829    5066 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322832    5066 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322834    5066 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322837    5066 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:31:36.336915 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322840    5066 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322843    5066 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322846    5066 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322848    5066 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322851    5066 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322853    5066 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322856    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322858    5066 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322861    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322863    5066 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322866    5066 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322868    5066 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322876    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322879    5066 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322882    5066 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322884    5066 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322887    5066 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322889    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322892    5066 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322894    5066 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:31:36.337449 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322898    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322900    5066 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322905    5066 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322908    5066 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322911    5066 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322914    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322916    5066 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.322919    5066 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.328783    5066 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.328931    5066 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.328953    5066 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.328958    5066 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329028    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329034    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329038    5066 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329041    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329044    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329046    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:31:36.338084 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329049    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329052    5066 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329054    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329057    5066 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329060    5066 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329062    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329066    5066 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329070    5066 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329074    5066 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329077    5066 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329079    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329082    5066 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329084    5066 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329087    5066 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329090    5066 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329093    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329097    5066 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329111    5066 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329114    5066 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329117    5066 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:31:36.338515 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329120    5066 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329122    5066 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329125    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329133    5066 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329136    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329139    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329143    5066 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329146    5066 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329150    5066 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329152    5066 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329155    5066 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329158    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329160    5066 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329163    5066 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329166    5066 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329168    5066 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329171    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329174    5066 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329176    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:31:36.339006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329179    5066 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329182    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329184    5066 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329187    5066 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329189    5066 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329192    5066 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329194    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329197    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329200    5066 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329203    5066 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329206    5066 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329208    5066 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329211    5066 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329214    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329216    5066 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329219    5066 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329222    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329225    5066 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329227    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329230    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:31:36.339463 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329233    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329235    5066 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329238    5066 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329240    5066 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329243    5066 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329246    5066 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329248    5066 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329251    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329253    5066 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329255    5066 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329258    5066 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329260    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329263    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329265    5066 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329269    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329272    5066 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329275    5066 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329277    5066 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329280    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:31:36.339950 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329283    5066 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329286    5066 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329288    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329291    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329293    5066 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329296    5066 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329298    5066 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329301    5066 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329304    5066 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329306    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329309    5066 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329312    5066 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329314    5066 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329317    5066 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329320    5066 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329322    5066 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329325    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329328    5066 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329330    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329333    5066 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:31:36.340394 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329335    5066 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329338    5066 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.329374    5066 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329546    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329552    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329555    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329558    5066 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329561    5066 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329565    5066 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329568    5066 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329571    5066 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329574    5066 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329577    5066 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329580    5066 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329583    5066 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329606    5066 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:31:36.340958 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329612    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329615    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329618    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329621    5066 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329623    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329626    5066 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329628    5066 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329631    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329633    5066 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329636    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329638    5066 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329641    5066 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329643    5066 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329646    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329649    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329651    5066 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329654    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329656    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329659    5066 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:31:36.341354 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329661    5066 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329664    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329666    5066 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329668    5066 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329671    5066 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329673    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329677    5066 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329680    5066 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329682    5066 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329684    5066 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329688    5066 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329690    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329693    5066 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329695    5066 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329697    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329700    5066 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329702    5066 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329705    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329708    5066 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329710    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:31:36.341843 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329713    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329716    5066 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329718    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329721    5066 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329723    5066 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329725    5066 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329728    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329730    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329733    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329735    5066 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329738    5066 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329740    5066 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329743    5066 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329745    5066 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329748    5066 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329752    5066 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329755    5066 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329758    5066 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329761    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:31:36.342319 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329764    5066 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329766    5066 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329769    5066 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329771    5066 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329774    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329776    5066 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329779    5066 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329781    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329783    5066 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329786    5066 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329788    5066 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329791    5066 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329793    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329796    5066 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329799    5066 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329801    5066 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329804    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329806    5066 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329809    5066 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:31:36.342822 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329811    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329814    5066 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329816    5066 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329818    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329821    5066 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329824    5066 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329827    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329829    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329832    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329834    5066 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329837    5066 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329839    5066 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329842    5066 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329845    5066 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329848    5066 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329850    5066 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:31:36.343364 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.329877    5066 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329910    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329914    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329916    5066 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329919    5066 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329922    5066 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329925    5066 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329927    5066 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329930    5066 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329933    5066 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329935    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329938    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329941    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329943    5066 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329946    5066 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329948    5066 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329951    5066 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329954    5066 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329956    5066 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329959    5066 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 05:31:36.344006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329961    5066 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329964    5066 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329968    5066 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329973    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329976    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329978    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329981    5066 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329983    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329986    5066 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329988    5066 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329991    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329994    5066 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329997    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.329999    5066 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330002    5066 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330004    5066 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330007    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330010    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330012    5066 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 05:31:36.344620 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330014    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330017    5066 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330019    5066 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330022    5066 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330024    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330027    5066 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330029    5066 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330032    5066 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330034    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330037    5066 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330040    5066 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330042    5066 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330045    5066 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330047    5066 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330049    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330052    5066 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330054    5066 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330057    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330060    5066 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330062    5066 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 05:31:36.345082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330065    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330067    5066 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330070    5066 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330072    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330075    5066 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330078    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330080    5066 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330083    5066 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330085    5066 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330087    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330090    5066 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330092    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330095    5066 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330100    5066 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330102    5066 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330105    5066 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330107    5066 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330110    5066 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330114    5066 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 05:31:36.345682 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330117    5066 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330120    5066 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330123    5066 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330126    5066 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330129    5066 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330131    5066 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330134    5066 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330136    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330139    5066 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330141    5066 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330144    5066 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330146    5066 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330149    5066 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330151    5066 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330154    5066 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330156    5066 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330159    5066 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330161    5066 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330164    5066 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 05:31:36.346132 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330167    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330170    5066 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330172    5066 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330174    5066 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330177    5066 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330179    5066 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330182    5066 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330186    5066 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330190    5066 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:36.330192    5066 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.330272    5066 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.331275    5066 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.331327    5066 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.331518    5066 server.go:1014] "Starting client certificate rotation"
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.331626    5066 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.331672    5066 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 05:25:55 +0000 UTC" deadline="2028-07-18 07:37:22.240206888 +0000 UTC"
Oct 30 05:31:36.346607 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.331699    5066 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15050h5m45.908510053s"
Oct 30 05:31:36.347003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.332010    5066 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 05:31:36.347003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.333179    5066 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 05:31:36.347003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.334535    5066 log.go:25] "Validated CRI v1 runtime API"
Oct 30 05:31:36.347003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.336266    5066 log.go:25] "Validated CRI v1 image API"
Oct 30 05:31:36.347003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.336983    5066 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 05:31:36.347003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.338942    5066 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9f603ae9-a2e0-42a9-a511-0cc6f2cf44ca:/dev/nvme0n1p3 d5daec70-9968-46eb-98d8-08897dc79571:/dev/nvme0n1p4]
Oct 30 05:31:36.347183 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.338957    5066 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/09fa66e9e0d8ffed1d2f7c8685697159fb6de7aac8500a5df10dfba45090ad1f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/09fa66e9e0d8ffed1d2f7c8685697159fb6de7aac8500a5df10dfba45090ad1f/userdata/shm major:0 minor:237 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0c11cc7919f86587553eabfc7f1e38ecf45ca69f6da0796ec26d59dd919f3a07/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0c11cc7919f86587553eabfc7f1e38ecf45ca69f6da0796ec26d59dd919f3a07/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1d626d33ff578d8387a5281f5d535b1fc8664015abaff1af922173eb580defd8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1d626d33ff578d8387a5281f5d535b1fc8664015abaff1af922173eb580defd8/userdata/shm major:0 minor:348 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/234f6ba211b434eafd446ebb3a0e1e812e1f7be58cab99d97cad92fefc541a2f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/234f6ba211b434eafd446ebb3a0e1e812e1f7be58cab99d97cad92fefc541a2f/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2ebca268a5ecc2180eb5e0c3d08af50a88603c4550cf29243ada2b9f338f6567/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2ebca268a5ecc2180eb5e0c3d08af50a88603c4550cf29243ada2b9f338f6567/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/31f8ce6b77279e97269bfa4540ec82a0dc92aa008f105d03ca961012c5ba0256/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/31f8ce6b77279e97269bfa4540ec82a0dc92aa008f105d03ca961012c5ba0256/userdata/shm major:0 minor:295 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/406513e1463071b6445938af507dd56bca6de79786a14a04ef7ddfc876858b95/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/406513e1463071b6445938af507dd56bca6de79786a14a04ef7ddfc876858b95/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/50eb74303b4645b568e99c96ad8fbeaa01b9f8cc49e117a8b391ccf8104207ad/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/50eb74303b4645b568e99c96ad8fbeaa01b9f8cc49e117a8b391ccf8104207ad/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6255a142722f21e56116020f8e3ac7cbb6f18f187ab0379bc79e9e62907c9e54/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6255a142722f21e56116020f8e3ac7cbb6f18f187ab0379bc79e9e62907c9e54/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6568dac5d31741f81717f7f963ea0942fc4319cfeaa204feeeb730ef5fb61d83/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6568dac5d31741f81717f7f963ea0942fc4319cfeaa204feeeb730ef5fb61d83/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/715a9a98d3babaded9b1ff31b402ad532a0dac86c53a91a4d5de0159a9a505f0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/715a9a98d3babaded9b1ff31b402ad532a0dac86c53a91a4d5de0159a9a505f0/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7aab8ee591997264797d019f8c032698f8de028a7bc62a3c490f0f7b524f8afa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7aab8ee591997264797d019f8c032698f8de028a7bc62a3c490f0f7b524f8afa/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8086d04205d43e965e252adecc780e5d80289a8081c06ab5e9c6436a3e51457a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8086d04205d43e965e252adecc780e5d80289a8081c06ab5e9c6436a3e51457a/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/81b6c9eea343a12127abee87744a659b4bbfc749aa6ac9f7ced7926d54a222bb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/81b6c9eea343a12127abee87744a659b4bbfc749aa6ac9f7ced7926d54a222bb/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/82e9d2dc6491dce1a0f7058d9ce6749095a22608155a988484a0c045de971af0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/82e9d2dc6491dce1a0f7058d9ce6749095a22608155a988484a0c045de971af0/userdata/shm major:0 minor:345 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8e306019d1f5901620e7be039d572cd059f104094e542380c8ba3f76b2bd0133/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8e306019d1f5901620e7be039d572cd059f104094e542380c8ba3f76b2bd0133/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9c2b138d5586b27e31708785a0909a45b2a2168eb682c7d19a46c73a9b094d64/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9c2b138d5586b27e31708785a0909a45b2a2168eb682c7d19a46c73a9b094d64/userdata/shm major:0 minor:354 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a5112a1b37f0d6d31f1c8c74496d6cc647a08bf2457a218f0b932b903ddd4728/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a5112a1b37f0d6d31f1c8c74496d6cc647a08bf2457a218f0b932b903ddd4728/userdata/shm major:0 minor:335 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/af946f0caace3ff447d5c528d8fa32b2c6f7164c1d924a788f9fd15115106166/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/af946f0caace3ff447d5c528d8fa32b2c6f7164c1d924a788f9fd15115106166/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c5f940a78d2aef49f815a2dd9f61a66a2c9124d79514816b4140449224db99d9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c5f940a78d2aef49f815a2dd9f61a66a2c9124d79514816b4140449224db99d9/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ca318e520dc7f05fc01002867cc54d58bc9e6382ea5e0c6ada15c4861c8f8ee0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ca318e520dc7f05fc01002867cc54d58bc9e6382ea5e0c6ada15c4861c8f8ee0/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ee0e9dd0e3fbb8cec8916714da03b17114aa8711de5614358f41153935f4eeef/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ee0e9dd0e3fbb8cec8916714da03b17114aa8711de5614358f41153935f4eeef/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f367b6ce0cb23d3c0cb99db8f880c07c8359f3bc27fd593149ba7f2d801f3ca4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f367b6ce0cb23d3c0cb99db8f880c07c8359f3bc27fd593149ba7f2d801f3ca4/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f3d5ba1a086c66afec5f3dfe4bbbc0781dabebdf88b9126df4a0dff0a7794d8a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f3d5ba1a086c66afec5f3dfe4bbbc0781dabebdf88b9126df4a0dff0a7794d8a/userdata/shm major:0 minor:241 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f99c68ea19609ad8745897cfdc054bbd985b7c73aa36064660d0e28411f36e63/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f99c68ea19609ad8745897cfdc054bbd985b7c73aa36064660d0e28411f36e63/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fb581939b0d830c6cf834034c0eae53edc3496352ef73bb25cfa79cf541cd4fd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fb581939b0d830c6cf834034c0eae53edc3496352ef73bb25cfa79cf541cd4fd/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02a30d07-33cc-4be0-a920-8e839f8a493b/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/02a30d07-33cc-4be0-a920-8e839f8a493b/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02a30d07-33cc-4be0-a920-8e839f8a493b/volumes/kubernetes.io~projected/kube-api-access-rznfw:{mountpoint:/var/lib/kubelet/pods/02a30d07-33cc-4be0-a920-8e839f8a493b/volumes/kubernetes.io~projected/kube-api-access-rznfw major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02a30d07-33cc-4be0-a920-8e839f8a493b/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/02a30d07-33cc-4be0-a920-8e839f8a493b/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0421f010-8e05-4896-b171-301e85964bb5/volumes/kubernetes.io~projected/kube-api-access-s7cjr:{mountpoint:/var/lib/kubelet/pods/0421f010-8e05-4896-b171-301e85964bb5/volumes/kubernetes.io~projected/kube-api-access-s7cjr major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06639c1a-71fb-47d6-898f-464546d8b66a/volumes/kubernetes.io~projected/kube-api-access-9l9nz:{mountpoint:/var/lib/kubelet/pods/06639c1a-71fb-47d6-898f-464546d8b66a/volumes/kubernetes.io~projected/kube-api-access-9l9nz major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06639c1a-71fb-47d6-898f-464546d8b66a/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/06639c1a-71fb-47d6-898f-464546d8b66a/volumes/kubernetes.io~secret/signing-key major:0 minor:344 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0cf86020-761f-4fbc-808e-7dc6c5d5515f/volumes/kubernetes.io~projected/kube-api-access-sv985:{mountpoint:/var/lib/kubelet/pods/0cf86020-761f-4fbc-808e-7dc6c5d5515f/volumes/kubernetes.io~projected/kube-api-access-sv985 major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0cf86020-761f-4fbc-808e-7dc6c5d5515f/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/0cf86020-761f-4fbc-808e-7dc6c5d5515f/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/14a30f61-33a2-4f29-ba24-fc02c0c6cc15/volumes/kubernetes.io~projected/kube-api-access-8rd8b:{mountpoint:/var/lib/kubelet/pods/14a30f61-33a2-4f29-ba24-fc02c0c6cc15/volumes/kubernetes.io~projected/kube-api-access-8rd8b major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c2414f4-fa94-4054-9b41-b264e6088e67/volumes/kubernetes.io~projected/kube-api-access-mrgd8:{mountpoint:/var/lib/kubelet/pods/1c2414f4-fa94-4054-9b41-b264e6088e67/volumes/kubernetes.io~projected/kube-api-access-mrgd8 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2f92fa2c-9133-484f-b326-a12b2b618900/volumes/kubernetes.io~projected/kube-api-access-ml8s5:{mountpoint:/var/lib/kubelet/pods/2f92fa2c-9133-484f-b326-a12b2b618900/volumes/kubernetes.io~projected/kube-api-access-ml8s5 major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/41edacc9-dea1-4343-96f5-5caf8a82fe84/volumes/kubernetes.io~projected/kube-api-access-9tmzj:{mountpoint:/var/lib/kubelet/pods/41edacc9-dea1-4343-96f5-5caf8a82fe84/volumes/kubernetes.io~projected/kube-api-access-9tmzj major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/41edacc9-dea1-4343-96f5-5caf8a82fe84/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/41edacc9-dea1-4343-96f5-5caf8a82fe84/volumes/kubernetes.io~secret/serving-cert major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4684e0e6-736f-41bf-8f22-77245e339d58/volumes/kubernetes.io~projected/kube-api-access-dffbj:{mountpoint:/var/lib/kubelet/pods/4684e0e6-736f-41bf-8f22-77245e339d58/volumes/kubernetes.io~projected/kube-api-access-dffbj major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4684e0e6-736f-41bf-8f22-77245e339d58/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/4684e0e6-736f-41bf-8f22-77245e339d58/volumes/kubernetes.io~secret/default-certificate major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4684e0e6-736f-41bf-8f22-77245e339d58/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/4684e0e6-736f-41bf-8f22-77245e339d58/volumes/kubernetes.io~secret/stats-auth major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46c256fd-a89b-41f9-95aa-d8e81f783820/volumes/kubernetes.io~projected/kube-api-access-rvb6d:{mountpoint:/var/lib/kubelet/pods/46c256fd-a89b-41f9-95aa-d8e81f783820/volumes/kubernetes.io~projected/kube-api-access-rvb6d major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/60bf362c-030f-4adf-9878-842b2fdc326b/volumes/kubernetes.io~projected/kube-api-access-44rjf:{mountpoint:/var/lib/kubelet/pods/60bf362c-030f-4adf-9878-842b2fdc326b/volumes/kubernetes.io~projected/kube-api-access-44rjf major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/666a74c6-3979-480a-b4d7-4872c84b2413/volumes/kubernetes.io~projected/kube-api-access-n7pkr:{mountpoint:/var/lib/kubelet/pods/666a74c6-3979-480a-b4d7-4872c84b2413/volumes/kubernetes.io~projected/kube-api-access-n7pkr major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7319282a-25bd-402e-a55c-8cc81da483c1/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/7319282a-25bd-402e-a55c-8cc81da483c1/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/834a0562-2972-449c-9ec4-a69908732c37/volumes/kubernetes.io~projected/kube-api-access-z9hln:{mountpoint:/var/lib/kubelet/pods/834a0562-2972-449c-9ec4-a69908732c37/volumes/kubernetes.io~projected/kube-api-access-z9hln major:0 minor:343 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/86fadfbe-f000-4b42-a8ff-7b67061959c5/volumes/kubernetes.io~projected/kube-api-access-jfzsh:{mountpoint:/var/lib/kubelet/pods/86fadfbe-f000-4b42-a8ff-7b67061959c5/volumes/kubernetes.io~projected/kube-api-access-jfzsh major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/86fadfbe-f000-4b42-a8ff-7b67061959c5/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/86fadfbe-f000-4b42-a8ff-7b67061959c5/volumes/kubernetes.io~secret/serving-cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a0ea8679-d95b-49d6-a614-9bb4eb4ae029/volumes/kubernetes.io~projected/kube-api-access-ct475:{mountpoint:/var/lib/kubelet/pods/a0ea8679-d95b-49d6-a614-9bb4eb4ae029/volumes/kubernetes.io~projected/kube-api-access-ct475 major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a655cfec-4ee3-4ac9-badc-4fb318603506/volumes/kubernetes.io~projected/kube-api-access-pq7d9:{mountpoint:/var/lib/kubelet/pods/a655cfec-4ee3-4ac9-badc-4fb318603506/volumes/kubernetes.io~projected/kube-api-access-pq7d9 major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a655cfec-4ee3-4ac9-badc-4fb318603506/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/a655cfec-4ee3-4ac9-badc-4fb318603506/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a6b6c8b1-dea3-45a1-b78f-676de9310b70/volumes/kubernetes.io~projected/kube-api-access-8zcgx:{mountpoint:/var/lib/kubelet/pods/a6b6c8b1-dea3-45a1-b78f-676de9310b70/volumes/kubernetes.io~projected/kube-api-access-8zcgx major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab824d2b-de9a-4384-a4a8-166f58808be8/volumes/kubernetes.io~projected/kube-api-access-xdz8s:{mountpoint:/var/lib/kubelet/pods/ab824d2b-de9a-4384-a4a8-166f58808be8/volumes/kubernetes.io~projected/kube-api-access-xdz8s major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab824d2b-de9a-4384-a4a8-166f58808be8/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/ab824d2b-de9a-4384-a4a8-166f58808be8/volumes/kubernetes.io~secret/serving-cert major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/abe28d56-457c-4e87-9cb1-804247d48da0/volumes/kubernetes.io~projected/kube-api-access-dwdkg:{mountpoint:/var/lib/kubelet/pods/abe28d56-457c-4e87-9cb1-804247d48da0/volumes/kubernetes.io~projected/kube-api-access-dwdkg major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b4c2b487-ec63-4b07-a11c-781cb9afd37d/volumes/kubernetes.io~projected/kube-api-access-77k6x:{mountpoint:/var/lib/kubelet/pods/b4c2b487-ec63-4b07-a11c-781cb9afd37d/volumes/kubernetes.io~projected/kube-api-access-77k6x major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bfa3e142-50e3-4b85-83fa-46f7dc8b7858/volumes/kubernetes.io~projected/kube-api-access-l72vm:{mountpoint:/var/lib/kubelet/pods/bfa3e142-50e3-4b85-83fa-46f7dc8b7858/volumes/kubernetes.io~projected/kube-api-access-l72vm major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c46db00b-3c48-4d8d-844c-918d8126cc41/volumes/kubernetes.io~projected/kube-api-access-7xdbt:{mountpoint:/var/lib/kubelet/pods/c46db00b-3c48-4d8d-844c-918d8126cc41/volumes/kubernetes.io~projected/kube-api-access-7xdbt major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166/volumes/kubernetes.io~projected/kube-api-access-96hw6:{mountpoint:/var/lib/kubelet/pods/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166/volumes/kubernetes.io~projected/kube-api-access-96hw6 major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes/kubernetes.io~projected/kube-api-access-wrchp:{mountpoint:/var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes/kubernetes.io~projected/kube-api-access-wrchp major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dfbf26d6-b6c8-4d33-9219-5e9fff25a103/volumes/kubernetes.io~projected/kube-api-access-8d4vl:{mountpoint:/var/lib/kubelet/pods/dfbf26d6-b6c8-4d33-9219-5e9fff25a103/volumes/kubernetes.io~projected/kube-api-access-8d4vl major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e71a9464-fdd4-491d-a0ce-51519a35486d/volumes/kubernetes.io~projected/kube-api-access-djp28:{mountpoint:/var/lib/kubelet/pods/e71a9464-fdd4-491d-a0ce-51519a35486d/volumes/kubernetes.io~projected/kube-api-access-djp28 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes/kubernetes.io~projected/kube-api-access-bptqj:{mountpoint:/var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes/kubernetes.io~projected/kube-api-access-bptqj major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ecfec034-dabf-45fb-a6cb-ef3ace810cb2/volumes/kubernetes.io~projected/kube-api-access-cvw56:{mountpoint:/var/lib/kubelet/pods/ecfec034-dabf-45fb-a6cb-ef3ace810cb2/volumes/kubernetes.io~projected/kube-api-access-cvw56 major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f74041a6-7409-439c-9ecb-4bc8dcc2006e/volumes/kubernetes.io~projected/kube-api-access-rfxnt:{mountpoint:/var/lib/kubelet/pods/f74041a6-7409-439c-9ecb-4bc8dcc2006e/volumes/kubernetes.io~projected/kube-api-access-rfxnt major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f82afdb2-9cea-479b-8d0b-88a814886c3a/volumes/kubernetes.io~projected/kube-api-access-5wkpm:{mountpoint:/var/lib/kubelet/pods/f82afdb2-9cea-479b-8d0b-88a814886c3a/volumes/kubernetes.io~projected/kube-api-access-5wkpm major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fcdb9d1a-474b-49f3-8683-f3874ad9d769/volumes/kubernetes.io~projected/kube-api-access-5m7s7:{mountpoint:/var/lib/kubelet/pods/fcdb9d1a-474b-49f3-8683-f3874ad9d769/volumes/kubernetes.io~projected/kube-api-access-5m7s7 major:0 minor:219 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/f8c861690d08c1c13dd53a06e059c33574497c90de9e335fb6a6dd6006f73a28/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/a74b378c0b4c9fc51a8e485691650c554d6b5873c68779526bad0357b3d23819/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/6612ccb2feea39298106e9337ecff0b6c8f3dd90d0f8b47c63520234cf74fdcf/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/93edce659438ddfa2839ebe6fa8507a47d6c012fe05994223bddd0562a4213b5/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/59cc946f8b6707b1f6df7b14033f5a78a699e903d5127b472605e3950eef8042/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/d6301c21c8b1c792233c44f9829aed5a1b4150bb7978c0b0e9aa2bb98a0ba137/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/eb49b098e2c58ca0dd6b63ccdc4c152897c38a011105b69bb8e69f3ec0ae95d7/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/b2d8621973523c7f48c74de1915e6786ff2f527b6be11c328f37428acbf9160f/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/742bafa44ae5d3ad8f54c9ea625730fe47ecaf01e2e936f871b8fae54b1ad079/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/2ce55e080838c791b4d966f16a9946919943e1c75629829e96b7b34beb85b961/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/ff22dc486662861f3615ec71a8a0e738f5fcdae6944c3d8387f27b03d08c6d3c/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/af2984bc094042c3bbf54624c52718d9f4ed8d99c505f481a8489d72d898b932/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/16b569774f47098fd7851fd98d393b79468b422116cc78afbc5df0f56cfc970a/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/c076dc66fea6e6265135f5499480999d6f7a712089ad0e437009cc5b720bd41e/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/0d2a5caec95883907c6db934475cb300d20724f5032ab5bcbc32556f1b891197/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/946fdfe8165b37c7fda7c46b41b7b34fa30b4001cdec86a6a077cd73ba492419/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/ea766e2aa82d0347a07a9e087d57e242282e98255d14f72172136e4ca0f0184c/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/5a9f914ea8cf25ad8976bc835479392cc72e5925c7a132795ee95cdcbed7f486/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/cf423040b32f44447a265275c68a2e09977ed9e377e634832d4e3a1603245866/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/38c259a60cbd7851548ad9673b24aa6e62e86ed32c9d1f175dd873526304e91b/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/283dc89ecc8ee480090830045c9a62618186230bfbf9074080bd887978e22c8a/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/2c43432b72b9c6cd71c9fe084a0b992f253847c1b17ed2842a32d1f9ee15ac10/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/f96251cdd2274d5973b23ee0844ed11dbf564cc94ddb0af8b7583c8d845b5647/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/f028c9ddd94e63fc33c1d07d54b93c21ec71eb272f6815d7c8a157b13b971447/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/5e4a67bc946230a1d62a1c8892a8c027a37c8775077896a2cb5811af5fbb3f3c/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/e69917b012570e7b327ca8dfa551085324215d29d054cfe2ac5a8645759486d0/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/7b3c16cf62e53e4c32c998a5e0e7fa8e8ca67953a82b809f47fc16ab10cefc04/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/48d27a57ea53454f269912ba5e94d2c0281966f613e19554181a23c2126267ef/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/b576eb6929d1097648e4925c14ab1ab67145233ef0629265974914623087fb89/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/da34c54d677e8ba71985bfa628d04386e52e30bb397321473dea47ebbdf11ea2/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/b02a7d22f95ec5881e16aba0b416b03296e27605959f9f9fe07deb300bde4184/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/3eee0fe0cd6824e8028414b99a7d5ce5009ddf1a78d37d1cc92a799a18e109b5/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/68aca56742b672ecd499587e04d32d1bb8efed8541666881218db2e41ebd8f74/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/a5936cfb0a07538d4b6eb3b1ef298e3253bccdcd1485a47f69e6b5f43273fbea/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/15c89fff6edceb5580c3d76011a36a2da05e8af4de17d048b042c0d37dd6d664/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/2efcdf03ea0e2f2735905fa9f8b7cab577d17264a0034c81a1cd9a2535aff481/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/f9b6d7e8cd40a791cefdafe792d7743abd2f389b64449d3b3a259c5e0aad3add/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/31d59fa8ba0936668c659e0a36a48bfdc9c2aa7d718b727d7b5d83be2fc3fe2f/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/9dd811fd90e10ce85b56b8690fee5b8af50538ad58f2203a446a0afc57150bb1/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/65fa72f0424b491542d420d7a4f14d5e7dc2afc27d41c43d81b9259876d3d6b3/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/e5f468caeb0fc2a0d3bfe439d3da03803a76d9e016037824c37eb0ea4cc5aec4/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/e4db5a16a359a36931e973e196848dbb2e51c371c76c4672609e64548ff9d693/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/121035ae9525cc000e7c61742a0f4c40790035b2d9994c52a72e69c86b6b48e7/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/66c2ea05470dc68b3b39a4fd6dbf7ff1ee38910e6f5e6a5e4000ee701ee11b92/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/dc03bf892d7bae646cb76c4b440d5ed631646773915167eaf4ae8ddcc35771b4/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-360:{mountpoint:/var/lib/containers/storage/overlay/583678322c78b9d653f0dc26693b7cf36c21cddea10e0839bddcc235d97476e0/merged major:0 minor:360 fsType:overlay blockSize:0} overlay_0-362:{mountpoint:/var/lib/containers/storage/overlay/3a472249f3aa562c6afd1f92de40a37b742e72736b6f61f2dbc99fec3bbbe08a/merged major:0 minor:362 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/f6e66c6b40aded09180fb65a76e3d36afb7efa3cedf67d47399ccd6cb84e8638/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/4e7736e0074cf779a4be9368f56709a794554f4bba8b1385f732bedb27afcccb/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/dd669d8dd8ff017a5c53406038b49d77774e369085815e0c2eacd03abfe30656/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/ec2ea01dc40465688702904d6f803a05302d1a2e7c2efdfb15951a55a5119dae/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/228b6aabd693e5c134fc5b272bf08632d9d8e5be6f28fcdaf24d9920f46d9576/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/15c0208115d8143b619a1ee59fb1b6da86f68e90c9f5c290f7681c56f1866a25/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/2283acdb10632a849965aaee91032a4f625ee7d8dd9c511b88321a3accf1ed8f/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/3c7eadce4cb5c198ebc0ef035c575e61b232ae401c64900320ec95ce2d3f9bc4/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/4f83079331ba89120c3547472eaf7134399510405f39c78f3e3eb7ee41db63fd/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-79:{mountpoint:/var/lib/containers/storage/overlay/7cdc8db2069cc4332f74b36e241b7e3cde5be1e6508350d30be82233fa020f71/merged major:0 minor:79 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/e167c00a44456c62009586639945d86c1b93f68511dd1b069d038c9d4d2efc73/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-85:{mountpoint:/var/lib/containers/storage/overlay/e7b762619dfb5788bbe7693d55ff47f2cee1a1e33eeade1620c1b15cb1709eb7/merged major:0 minor:85 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/0c91c312412c00772eb0bd6ec4a463e088a63d50d06396566e3ee87b2c6d7abb/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/92272e87ce86b89ee14b15a848df4eebc2613973cd5662322bb250d30f8e3e97/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-97:{mountpoint:/var/lib/containers/storage/overlay/65f24d4be04fe5e8f2bd6a35b36a7599ae60a3ec0699bff35b2dd4fa1430dc1f/merged major:0 minor:97 fsType:overlay blockSize:0}]
Oct 30 05:31:36.348305 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.347954    5066 manager.go:217] Machine: {Timestamp:2026-10-30 05:31:36.3474807 +0000 UTC m=+0.092766023 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100161 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2823b3e9408d8f6a0bf9d2e392ec18 SystemUUID:ec2823b3-e940-8d8f-6a0b-f9d2e392ec18 BootID:89340346-a628-40d7-99f1-fbcfd93ab3f1 Filesystems:[{Device:/var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes/kubernetes.io~projected/kube-api-access-bptqj DeviceMajor:0 DeviceMinor:215 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6255a142722f21e56116020f8e3ac7cbb6f18f187ab0379bc79e9e62907c9e54/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-79 DeviceMajor:0 DeviceMinor:79 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/834a0562-2972-449c-9ec4-a69908732c37/volumes/kubernetes.io~projected/kube-api-access-z9hln DeviceMajor:0 DeviceMinor:343 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/02a30d07-33cc-4be0-a920-8e839f8a493b/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/a0ea8679-d95b-49d6-a614-9bb4eb4ae029/volumes/kubernetes.io~projected/kube-api-access-ct475 DeviceMajor:0 DeviceMinor:220 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1d626d33ff578d8387a5281f5d535b1fc8664015abaff1af922173eb580defd8/userdata/shm DeviceMajor:0 DeviceMinor:348 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06639c1a-71fb-47d6-898f-464546d8b66a/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:344 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/02a30d07-33cc-4be0-a920-8e839f8a493b/volumes/kubernetes.io~projected/kube-api-access-rznfw DeviceMajor:0 DeviceMinor:66 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2f92fa2c-9133-484f-b326-a12b2b618900/volumes/kubernetes.io~projected/kube-api-access-ml8s5 DeviceMajor:0 DeviceMinor:224 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/46c256fd-a89b-41f9-95aa-d8e81f783820/volumes/kubernetes.io~projected/kube-api-access-rvb6d DeviceMajor:0 DeviceMinor:226 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4684e0e6-736f-41bf-8f22-77245e339d58/volumes/kubernetes.io~projected/kube-api-access-dffbj DeviceMajor:0 DeviceMinor:227 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8086d04205d43e965e252adecc780e5d80289a8081c06ab5e9c6436a3e51457a/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7319282a-25bd-402e-a55c-8cc81da483c1/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0421f010-8e05-4896-b171-301e85964bb5/volumes/kubernetes.io~projected/kube-api-access-s7cjr DeviceMajor:0 DeviceMinor:64 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a655cfec-4ee3-4ac9-badc-4fb318603506/volumes/kubernetes.io~projected/kube-api-access-pq7d9 DeviceMajor:0 DeviceMinor:214 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/31f8ce6b77279e97269bfa4540ec82a0dc92aa008f105d03ca961012c5ba0256/userdata/shm DeviceMajor:0 DeviceMinor:295 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fb581939b0d830c6cf834034c0eae53edc3496352ef73bb25cfa79cf541cd4fd/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0cf86020-761f-4fbc-808e-7dc6c5d5515f/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-360 DeviceMajor:0 DeviceMinor:360 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/406513e1463071b6445938af507dd56bca6de79786a14a04ef7ddfc876858b95/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a655cfec-4ee3-4ac9-badc-4fb318603506/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4684e0e6-736f-41bf-8f22-77245e339d58/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:204 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c46db00b-3c48-4d8d-844c-918d8126cc41/volumes/kubernetes.io~projected/kube-api-access-7xdbt DeviceMajor:0 DeviceMinor:218 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:288 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/abe28d56-457c-4e87-9cb1-804247d48da0/volumes/kubernetes.io~projected/kube-api-access-dwdkg DeviceMajor:0 DeviceMinor:342 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6568dac5d31741f81717f7f963ea0942fc4319cfeaa204feeeb730ef5fb61d83/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4684e0e6-736f-41bf-8f22-77245e339d58/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:45 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/dfbf26d6-b6c8-4d33-9219-5e9fff25a103/volumes/kubernetes.io~projected/kube-api-access-8d4vl DeviceMajor:0 DeviceMinor:211 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ab824d2b-de9a-4384-a4a8-166f58808be8/volumes/kubernetes.io~projected/kube-api-access-xdz8s DeviceMajor:0 DeviceMinor:221 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a6b6c8b1-dea3-45a1-b78f-676de9310b70/volumes/kubernetes.io~projected/kube-api-access-8zcgx DeviceMajor:0 DeviceMinor:62 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0cf86020-761f-4fbc-808e-7dc6c5d5515f/volumes/kubernetes.io~projected/kube-api-access-sv985 DeviceMajor:0 DeviceMinor:222 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f367b6ce0cb23d3c0cb99db8f880c07c8359f3bc27fd593149ba7f2d801f3ca4/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-97 DeviceMajor:0 DeviceMinor:97 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/09fa66e9e0d8ffed1d2f7c8685697159fb6de7aac8500a5df10dfba45090ad1f/userdata/shm DeviceMajor:0 DeviceMinor:237 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/715a9a98d3babaded9b1ff31b402ad532a0dac86c53a91a4d5de0159a9a505f0/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9c2b138d5586b27e31708785a0909a45b2a2168eb682c7d19a46c73a9b094d64/userdata/shm DeviceMajor:0 DeviceMinor:354 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f99c68ea19609ad8745897cfdc054bbd985b7c73aa36064660d0e28411f36e63/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/82e9d2dc6491dce1a0f7058d9ce6749095a22608155a988484a0c045de971af0/userdata/shm DeviceMajor:0 DeviceMinor:345 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/60bf362c-030f-4adf-9878-842b2fdc326b/volumes/kubernetes.io~projected/kube-api-access-44rjf DeviceMajor:0 DeviceMinor:63 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1c2414f4-fa94-4054-9b41-b264e6088e67/volumes/kubernetes.io~projected/kube-api-access-mrgd8 DeviceMajor:0 DeviceMinor:67 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/af946f0caace3ff447d5c528d8fa32b2c6f7164c1d924a788f9fd15115106166/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:203 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/50eb74303b4645b568e99c96ad8fbeaa01b9f8cc49e117a8b391ccf8104207ad/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7aab8ee591997264797d019f8c032698f8de028a7bc62a3c490f0f7b524f8afa/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/86fadfbe-f000-4b42-a8ff-7b67061959c5/volumes/kubernetes.io~projected/kube-api-access-jfzsh DeviceMajor:0 DeviceMinor:210 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b4c2b487-ec63-4b07-a11c-781cb9afd37d/volumes/kubernetes.io~projected/kube-api-access-77k6x DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166/volumes/kubernetes.io~projected/kube-api-access-96hw6 DeviceMajor:0 DeviceMinor:61 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ee0e9dd0e3fbb8cec8916714da03b17114aa8711de5614358f41153935f4eeef/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-85 DeviceMajor:0 DeviceMinor:85 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2ebca268a5ecc2180eb5e0c3d08af50a88603c4550cf29243ada2b9f338f6567/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ca318e520dc7f05fc01002867cc54d58bc9e6382ea5e0c6ada15c4861c8f8ee0/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:209 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/86fadfbe-f000-4b42-a8ff-7b67061959c5/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:197 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:216 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes/kubernetes.io~projected/kube-api-access-wrchp DeviceMajor:0 DeviceMinor:217 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-362 DeviceMajor:0 DeviceMinor:362 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f82afdb2-9cea-479b-8d0b-88a814886c3a/volumes/kubernetes.io~projected/kube-api-access-5wkpm DeviceMajor:0 DeviceMinor:293 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ab824d2b-de9a-4384-a4a8-166f58808be8/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:202 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:223 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/14a30f61-33a2-4f29-ba24-fc02c0c6cc15/volumes/kubernetes.io~projected/kube-api-access-8rd8b DeviceMajor:0 DeviceMinor:65 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0c11cc7919f86587553eabfc7f1e38ecf45ca69f6da0796ec26d59dd919f3a07/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:208 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/fcdb9d1a-474b-49f3-8683-f3874ad9d769/volumes/kubernetes.io~projected/kube-api-access-5m7s7 DeviceMajor:0 DeviceMinor:219 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/81b6c9eea343a12127abee87744a659b4bbfc749aa6ac9f7ced7926d54a222bb/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/41edacc9-dea1-4343-96f5-5caf8a82fe84/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:205 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/666a74c6-3979-480a-b4d7-4872c84b2413/volumes/kubernetes.io~projected/kube-api-access-n7pkr DeviceMajor:0 DeviceMinor:213 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/41edacc9-dea1-4343-96f5-5caf8a82fe84/volumes/kubernetes.io~projected/kube-api-access-9tmzj DeviceMajor:0 DeviceMinor:225 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ecfec034-dabf-45fb-a6cb-ef3ace810cb2/volumes/kubernetes.io~projected/kube-api-access-cvw56 DeviceMajor:0 DeviceMinor:334 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/06639c1a-71fb-47d6-898f-464546d8b66a/volumes/kubernetes.io~projected/kube-api-access-9l9nz DeviceMajor:0 DeviceMinor:347 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8e306019d1f5901620e7be039d572cd059f104094e542380c8ba3f76b2bd0133/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a5112a1b37f0d6d31f1c8c74496d6cc647a08bf2457a218f0b932b903ddd4728/userdata/shm DeviceMajor:0 DeviceMinor:335 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/234f6ba211b434eafd446ebb3a0e1e812e1f7be58cab99d97cad92fefc541a2f/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f3d5ba1a086c66afec5f3dfe4bbbc0781dabebdf88b9126df4a0dff0a7794d8a/userdata/shm DeviceMajor:0 DeviceMinor:241 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f74041a6-7409-439c-9ecb-4bc8dcc2006e/volumes/kubernetes.io~projected/kube-api-access-rfxnt DeviceMajor:0 DeviceMinor:337 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/e71a9464-fdd4-491d-a0ce-51519a35486d/volumes/kubernetes.io~projected/kube-api-access-djp28 DeviceMajor:0 DeviceMinor:60 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bfa3e142-50e3-4b85-83fa-46f7dc8b7858/volumes/kubernetes.io~projected/kube-api-access-l72vm DeviceMajor:0 DeviceMinor:212 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/02a30d07-33cc-4be0-a920-8e839f8a493b/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c5f940a78d2aef49f815a2dd9f61a66a2c9124d79514816b4140449224db99d9/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:207 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:09fa66e9e0d8ffe MacAddress:1e:92:04:74:b3:41 Speed:10000 Mtu:8901} {Name:1d626d33ff578d8 MacAddress:a6:a3:e8:20:2a:31 Speed:10000 Mtu:8901} {Name:2ebca268a5ecc21 MacAddress:e2:dd:0d:62:db:6d Speed:10000 Mtu:8901} {Name:31f8ce6b77279e9 MacAddress:f6:28:f3:d8:1c:09 Speed:10000 Mtu:8901} {Name:50eb74303b4645b MacAddress:fe:7e:0c:5c:d8:07 Speed:10000 Mtu:8901} {Name:715a9a98d3babad MacAddress:e2:6c:c9:c0:94:8a Speed:10000 Mtu:8901} {Name:7aab8ee59199726 MacAddress:ee:fb:05:e3:73:d5 Speed:10000 Mtu:8901} {Name:8086d04205d43e9 MacAddress:be:98:31:44:52:cc Speed:10000 Mtu:8901} {Name:82e9d2dc6491dce MacAddress:46:7a:ea:43:5a:98 Speed:10000 Mtu:8901} {Name:8e306019d1f5901 MacAddress:7a:20:eb:89:2d:f7 Speed:10000 Mtu:8901} {Name:9c2b138d5586b27 MacAddress:76:f3:4e:50:24:3a Speed:10000 Mtu:8901} {Name:a5112a1b37f0d6d MacAddress:aa:d5:ca:3d:ef:90 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:ed:0b:9a:87 Speed:0 Mtu:9001} {Name:br-int MacAddress:ea:0e:fd:f1:f8:f9 Speed:0 Mtu:8901} {Name:ca318e520dc7f05 MacAddress:ce:2d:2f:76:c2:ae Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:ed:0b:9a:87 Speed:0 Mtu:9001} {Name:f367b6ce0cb23d3 MacAddress:92:32:13:3c:bd:52 Speed:10000 Mtu:8901} {Name:f3d5ba1a086c66a MacAddress:ee:b1:67:e2:e3:5f Speed:10000 Mtu:8901} {Name:f99c68ea19609ad MacAddress:d2:39:76:85:9d:ed Speed:10000 Mtu:8901} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:5a:6d:98:2c:4f:34 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 05:31:36.348566 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.348309    5066 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 05:31:36.348566 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.348351    5066 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 05:31:36.348665 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.348642    5066 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 05:31:36.348814 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.348666    5066 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-173.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 05:31:36.348854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.348817    5066 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 05:31:36.348854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.348825    5066 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 05:31:36.348854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.348838    5066 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 05:31:36.348933 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.348857    5066 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 05:31:36.348960 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.348943    5066 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 05:31:36.349110 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.349093    5066 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 05:31:36.349222 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.349213    5066 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 05:31:36.349260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.349250    5066 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 05:31:36.349260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.349260    5066 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 05:31:36.349343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.349269    5066 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 05:31:36.349343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.349278    5066 kubelet.go:405] "Adding apiserver pod source"
Oct 30 05:31:36.349343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.349286    5066 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 05:31:36.350170 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.350149    5066 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 05:31:36.350170 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.350166    5066 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 05:31:36.350684 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.350665    5066 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 05:31:36.352836 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.352820    5066 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 05:31:36.352836 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.352832    5066 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 05:31:36.352980 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.352969    5066 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 05:31:36.353173 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353162    5066 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 05:31:36.353205 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353192    5066 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 05:31:36.353306 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353297    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 05:31:36.353350 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353311    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 05:31:36.353350 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353318    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 05:31:36.353350 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353323    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 05:31:36.353350 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353329    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 05:31:36.353350 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353340    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 05:31:36.353350 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353349    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 05:31:36.353568 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353357    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 05:31:36.353568 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353364    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 05:31:36.353568 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353370    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 05:31:36.353568 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353378    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 05:31:36.353568 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353387    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 05:31:36.353568 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353410    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 05:31:36.353568 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.353417    5066 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 05:31:36.357115 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.357081    5066 server.go:1263] "Started kubelet"
Oct 30 05:31:36.357287 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.357210    5066 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 05:31:36.357366 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.357313    5066 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 05:31:36.357366 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.357320    5066 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 05:31:36.358607 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.358504    5066 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 05:31:36.358626 ip-10-0-136-173 systemd[1]: Started Kubernetes Kubelet.
Oct 30 05:31:36.359528 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.359499    5066 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 05:31:36.363997 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.363944    5066 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 05:31:36.363997 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.363974    5066 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 05:31:36.364175 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.364158    5066 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 05:31:36.364175 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.364144    5066 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 05:25:57 +0000 UTC" deadline="2028-06-11 08:12:19.825526597 +0000 UTC"
Oct 30 05:31:36.364269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.364179    5066 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 05:31:36.364269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.364183    5066 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14162h40m43.46134662s"
Oct 30 05:31:36.364269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.364174    5066 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 05:31:36.364464 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:36.364318    5066 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-173.ec2.internal\" not found"
Oct 30 05:31:36.366130 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.366112    5066 factory.go:55] Registering systemd factory
Oct 30 05:31:36.366214 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.366133    5066 factory.go:223] Registration of the systemd container factory successfully
Oct 30 05:31:36.366508 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.366491    5066 factory.go:153] Registering CRI-O factory
Oct 30 05:31:36.366567 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.366513    5066 factory.go:223] Registration of the crio container factory successfully
Oct 30 05:31:36.366643 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.366571    5066 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 05:31:36.366643 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.366608    5066 factory.go:103] Registering Raw factory
Oct 30 05:31:36.366643 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.366624    5066 manager.go:1199] Started watching for new ooms in manager
Oct 30 05:31:36.367322 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367287    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02a30d07-33cc-4be0-a920-8e839f8a493b" volumeName="kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-ovnkube-config" seLinuxMountContext=""
Oct 30 05:31:36.367322 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367321    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06639c1a-71fb-47d6-898f-464546d8b66a" volumeName="kubernetes.io/secret/06639c1a-71fb-47d6-898f-464546d8b66a-signing-key" seLinuxMountContext=""
Oct 30 05:31:36.367445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367334    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4684e0e6-736f-41bf-8f22-77245e339d58" volumeName="kubernetes.io/projected/4684e0e6-736f-41bf-8f22-77245e339d58-kube-api-access-dffbj" seLinuxMountContext=""
Oct 30 05:31:36.367445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367347    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4684e0e6-736f-41bf-8f22-77245e339d58" volumeName="kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-default-certificate" seLinuxMountContext=""
Oct 30 05:31:36.367445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367385    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4684e0e6-736f-41bf-8f22-77245e339d58" volumeName="kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-stats-auth" seLinuxMountContext=""
Oct 30 05:31:36.367445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367398    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dac2ba5d-8990-47e3-aa65-02ff7500ac01" volumeName="kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 05:31:36.367445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367410    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02a30d07-33cc-4be0-a920-8e839f8a493b" volumeName="kubernetes.io/projected/02a30d07-33cc-4be0-a920-8e839f8a493b-kube-api-access-rznfw" seLinuxMountContext=""
Oct 30 05:31:36.367445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367423    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02a30d07-33cc-4be0-a920-8e839f8a493b" volumeName="kubernetes.io/secret/02a30d07-33cc-4be0-a920-8e839f8a493b-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 05:31:36.367445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367438    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06639c1a-71fb-47d6-898f-464546d8b66a" volumeName="kubernetes.io/projected/06639c1a-71fb-47d6-898f-464546d8b66a-kube-api-access-9l9nz" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367451    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41edacc9-dea1-4343-96f5-5caf8a82fe84" volumeName="kubernetes.io/configmap/41edacc9-dea1-4343-96f5-5caf8a82fe84-config" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367465    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7319282a-25bd-402e-a55c-8cc81da483c1" volumeName="kubernetes.io/secret/7319282a-25bd-402e-a55c-8cc81da483c1-agent-certs" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367477    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a655cfec-4ee3-4ac9-badc-4fb318603506" volumeName="kubernetes.io/projected/a655cfec-4ee3-4ac9-badc-4fb318603506-kube-api-access-pq7d9" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367490    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c46db00b-3c48-4d8d-844c-918d8126cc41" volumeName="kubernetes.io/projected/c46db00b-3c48-4d8d-844c-918d8126cc41-kube-api-access-7xdbt" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367512    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d432e7e1-19d3-4f6d-8d47-51f8d6a3d166" volumeName="kubernetes.io/projected/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-kube-api-access-96hw6" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367539    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dac2ba5d-8990-47e3-aa65-02ff7500ac01" volumeName="kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-certificates" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367551    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dac2ba5d-8990-47e3-aa65-02ff7500ac01" volumeName="kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-kube-api-access-wrchp" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367564    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" volumeName="kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367576    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06639c1a-71fb-47d6-898f-464546d8b66a" volumeName="kubernetes.io/configmap/06639c1a-71fb-47d6-898f-464546d8b66a-signing-cabundle" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367606    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="86fadfbe-f000-4b42-a8ff-7b67061959c5" volumeName="kubernetes.io/projected/86fadfbe-f000-4b42-a8ff-7b67061959c5-kube-api-access-jfzsh" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367617    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfa3e142-50e3-4b85-83fa-46f7dc8b7858" volumeName="kubernetes.io/projected/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-kube-api-access-l72vm" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367628    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dac2ba5d-8990-47e3-aa65-02ff7500ac01" volumeName="kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-installation-pull-secrets" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367639    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dfbf26d6-b6c8-4d33-9219-5e9fff25a103" volumeName="kubernetes.io/projected/dfbf26d6-b6c8-4d33-9219-5e9fff25a103-kube-api-access-8d4vl" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367651    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f74041a6-7409-439c-9ecb-4bc8dcc2006e" volumeName="kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-service-ca-bundle" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367664    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0cf86020-761f-4fbc-808e-7dc6c5d5515f" volumeName="kubernetes.io/secret/0cf86020-761f-4fbc-808e-7dc6c5d5515f-serving-cert" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367675    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="86fadfbe-f000-4b42-a8ff-7b67061959c5" volumeName="kubernetes.io/configmap/86fadfbe-f000-4b42-a8ff-7b67061959c5-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367685    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" volumeName="kubernetes.io/empty-dir/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-ca-trust-extracted" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367700    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f74041a6-7409-439c-9ecb-4bc8dcc2006e" volumeName="kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367712    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02a30d07-33cc-4be0-a920-8e839f8a493b" volumeName="kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367732    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0cf86020-761f-4fbc-808e-7dc6c5d5515f" volumeName="kubernetes.io/configmap/0cf86020-761f-4fbc-808e-7dc6c5d5515f-config" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367750    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6b6c8b1-dea3-45a1-b78f-676de9310b70" volumeName="kubernetes.io/projected/a6b6c8b1-dea3-45a1-b78f-676de9310b70-kube-api-access-8zcgx" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367759    5066 manager.go:319] Starting recovery of all containers
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367762    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e71a9464-fdd4-491d-a0ce-51519a35486d" volumeName="kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-cni-binary-copy" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367774    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e71a9464-fdd4-491d-a0ce-51519a35486d" volumeName="kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367791    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" volumeName="kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-kube-api-access-bptqj" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367804    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f74041a6-7409-439c-9ecb-4bc8dcc2006e" volumeName="kubernetes.io/empty-dir/f74041a6-7409-439c-9ecb-4bc8dcc2006e-archives-path" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367816    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f82afdb2-9cea-479b-8d0b-88a814886c3a" volumeName="kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm" seLinuxMountContext=""
Oct 30 05:31:36.367819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367828    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02a30d07-33cc-4be0-a920-8e839f8a493b" volumeName="kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-env-overrides" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367840    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c2414f4-fa94-4054-9b41-b264e6088e67" volumeName="kubernetes.io/configmap/1c2414f4-fa94-4054-9b41-b264e6088e67-cni-binary-copy" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367852    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46c256fd-a89b-41f9-95aa-d8e81f783820" volumeName="kubernetes.io/projected/46c256fd-a89b-41f9-95aa-d8e81f783820-kube-api-access-rvb6d" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367866    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="666a74c6-3979-480a-b4d7-4872c84b2413" volumeName="kubernetes.io/projected/666a74c6-3979-480a-b4d7-4872c84b2413-kube-api-access-n7pkr" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367878    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7319282a-25bd-402e-a55c-8cc81da483c1" volumeName="kubernetes.io/configmap/7319282a-25bd-402e-a55c-8cc81da483c1-konnectivity-ca" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367902    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa" volumeName="kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367915    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a655cfec-4ee3-4ac9-badc-4fb318603506" volumeName="kubernetes.io/secret/a655cfec-4ee3-4ac9-badc-4fb318603506-cert" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367928    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e71a9464-fdd4-491d-a0ce-51519a35486d" volumeName="kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367939    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b0bdcd7b-f471-405a-87b2-ebedb8a1c950" volumeName="kubernetes.io/configmap/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-nginx-conf" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367953    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bfa3e142-50e3-4b85-83fa-46f7dc8b7858" volumeName="kubernetes.io/configmap/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-telemetry-config" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367965    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14a30f61-33a2-4f29-ba24-fc02c0c6cc15" volumeName="kubernetes.io/projected/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-kube-api-access-8rd8b" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367977    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14a30f61-33a2-4f29-ba24-fc02c0c6cc15" volumeName="kubernetes.io/configmap/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-serviceca" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.367989    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f92fa2c-9133-484f-b326-a12b2b618900" volumeName="kubernetes.io/projected/2f92fa2c-9133-484f-b326-a12b2b618900-kube-api-access-ml8s5" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368000    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46c256fd-a89b-41f9-95aa-d8e81f783820" volumeName="kubernetes.io/configmap/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-config" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368013    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="834a0562-2972-449c-9ec4-a69908732c37" volumeName="kubernetes.io/projected/834a0562-2972-449c-9ec4-a69908732c37-kube-api-access-z9hln" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368027    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="abe28d56-457c-4e87-9cb1-804247d48da0" volumeName="kubernetes.io/projected/abe28d56-457c-4e87-9cb1-804247d48da0-kube-api-access-dwdkg" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368038    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="86fadfbe-f000-4b42-a8ff-7b67061959c5" volumeName="kubernetes.io/empty-dir/86fadfbe-f000-4b42-a8ff-7b67061959c5-tmp" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368051    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6b6c8b1-dea3-45a1-b78f-676de9310b70" volumeName="kubernetes.io/configmap/a6b6c8b1-dea3-45a1-b78f-676de9310b70-iptables-alerter-script" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368062    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab824d2b-de9a-4384-a4a8-166f58808be8" volumeName="kubernetes.io/configmap/ab824d2b-de9a-4384-a4a8-166f58808be8-config" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368073    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dac2ba5d-8990-47e3-aa65-02ff7500ac01" volumeName="kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-trusted-ca" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368086    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dac2ba5d-8990-47e3-aa65-02ff7500ac01" volumeName="kubernetes.io/empty-dir/dac2ba5d-8990-47e3-aa65-02ff7500ac01-ca-trust-extracted" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368121    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" volumeName="kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-installation-pull-secrets" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368134    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dac2ba5d-8990-47e3-aa65-02ff7500ac01" volumeName="kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-bound-sa-token" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368153    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" volumeName="kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-bound-sa-token" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368165    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ecfec034-dabf-45fb-a6cb-ef3ace810cb2" volumeName="kubernetes.io/projected/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-api-access-cvw56" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368177    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fcdb9d1a-474b-49f3-8683-f3874ad9d769" volumeName="kubernetes.io/projected/fcdb9d1a-474b-49f3-8683-f3874ad9d769-kube-api-access-5m7s7" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368188    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab824d2b-de9a-4384-a4a8-166f58808be8" volumeName="kubernetes.io/configmap/ab824d2b-de9a-4384-a4a8-166f58808be8-trusted-ca" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368199    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c46db00b-3c48-4d8d-844c-918d8126cc41" volumeName="kubernetes.io/configmap/c46db00b-3c48-4d8d-844c-918d8126cc41-config-volume" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368211    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d432e7e1-19d3-4f6d-8d47-51f8d6a3d166" volumeName="kubernetes.io/empty-dir/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-tuned" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368223    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0421f010-8e05-4896-b171-301e85964bb5" volumeName="kubernetes.io/projected/0421f010-8e05-4896-b171-301e85964bb5-kube-api-access-s7cjr" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368235    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41edacc9-dea1-4343-96f5-5caf8a82fe84" volumeName="kubernetes.io/projected/41edacc9-dea1-4343-96f5-5caf8a82fe84-kube-api-access-9tmzj" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368246    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60bf362c-030f-4adf-9878-842b2fdc326b" volumeName="kubernetes.io/projected/60bf362c-030f-4adf-9878-842b2fdc326b-kube-api-access-44rjf" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368259    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="86fadfbe-f000-4b42-a8ff-7b67061959c5" volumeName="kubernetes.io/secret/86fadfbe-f000-4b42-a8ff-7b67061959c5-serving-cert" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368275    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a0ea8679-d95b-49d6-a614-9bb4eb4ae029" volumeName="kubernetes.io/projected/a0ea8679-d95b-49d6-a614-9bb4eb4ae029-kube-api-access-ct475" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368287    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab824d2b-de9a-4384-a4a8-166f58808be8" volumeName="kubernetes.io/projected/ab824d2b-de9a-4384-a4a8-166f58808be8-kube-api-access-xdz8s" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368300    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4c2b487-ec63-4b07-a11c-781cb9afd37d" volumeName="kubernetes.io/empty-dir/b4c2b487-ec63-4b07-a11c-781cb9afd37d-tmp-dir" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368311    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d432e7e1-19d3-4f6d-8d47-51f8d6a3d166" volumeName="kubernetes.io/empty-dir/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-tmp" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368323    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c2414f4-fa94-4054-9b41-b264e6088e67" volumeName="kubernetes.io/configmap/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-daemon-config" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368334    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" volumeName="kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-trusted-ca" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368347    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f74041a6-7409-439c-9ecb-4bc8dcc2006e" volumeName="kubernetes.io/projected/f74041a6-7409-439c-9ecb-4bc8dcc2006e-kube-api-access-rfxnt" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368358    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41edacc9-dea1-4343-96f5-5caf8a82fe84" volumeName="kubernetes.io/secret/41edacc9-dea1-4343-96f5-5caf8a82fe84-serving-cert" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368375    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="86fadfbe-f000-4b42-a8ff-7b67061959c5" volumeName="kubernetes.io/configmap/86fadfbe-f000-4b42-a8ff-7b67061959c5-service-ca-bundle" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368387    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab824d2b-de9a-4384-a4a8-166f58808be8" volumeName="kubernetes.io/secret/ab824d2b-de9a-4384-a4a8-166f58808be8-serving-cert" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368398    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" volumeName="kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-certificates" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368410    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0cf86020-761f-4fbc-808e-7dc6c5d5515f" volumeName="kubernetes.io/projected/0cf86020-761f-4fbc-808e-7dc6c5d5515f-kube-api-access-sv985" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368424    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c2414f4-fa94-4054-9b41-b264e6088e67" volumeName="kubernetes.io/projected/1c2414f4-fa94-4054-9b41-b264e6088e67-kube-api-access-mrgd8" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368436    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="86fadfbe-f000-4b42-a8ff-7b67061959c5" volumeName="kubernetes.io/empty-dir/86fadfbe-f000-4b42-a8ff-7b67061959c5-snapshots" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368448    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e71a9464-fdd4-491d-a0ce-51519a35486d" volumeName="kubernetes.io/projected/e71a9464-fdd4-491d-a0ce-51519a35486d-kube-api-access-djp28" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368460    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4c2b487-ec63-4b07-a11c-781cb9afd37d" volumeName="kubernetes.io/projected/b4c2b487-ec63-4b07-a11c-781cb9afd37d-kube-api-access-77k6x" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368474    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c46db00b-3c48-4d8d-844c-918d8126cc41" volumeName="kubernetes.io/empty-dir/c46db00b-3c48-4d8d-844c-918d8126cc41-tmp-dir" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368486    5066 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ecfec034-dabf-45fb-a6cb-ef3ace810cb2" volumeName="kubernetes.io/empty-dir/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-data-volume" seLinuxMountContext=""
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368498    5066 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 05:31:36.369157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.368505    5066 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 05:31:36.383554 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.383530    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:31:36.383988 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.383967    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:31:36.392276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.392223    5066 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 05:31:36.393794 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.393770    5066 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 05:31:36.393794 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.393790    5066 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 05:31:36.393936 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.393811    5066 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 05:31:36.393936 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:36.393849    5066 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 05:31:36.409475 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.409457    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:31:36.409569 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.409481    5066 generic.go:356] "Generic (PLEG): container finished" podID="ab824d2b-de9a-4384-a4a8-166f58808be8" containerID="b6927366bfdc148db909b2ffc636d98277afcb80bc1db53d386c12c71c939fb8" exitCode=255
Oct 30 05:31:36.417140 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.417112    5066 generic.go:356] "Generic (PLEG): container finished" podID="6f17565975b3f18442fbd67650b5b56f" containerID="b5cbaa1666f97cc808001b56e15afbb07cf03552c94130d44e6d4c321f85f74d" exitCode=0
Oct 30 05:31:36.419900 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.419881    5066 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 05:31:36.419995 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.419938    5066 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 05:31:36.420857 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.420840    5066 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 05:31:36.435916 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.435834    5066 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 05:31:36.435916 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.435873    5066 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 05:31:36.459070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.459035    5066 generic.go:356] "Generic (PLEG): container finished" podID="e71a9464-fdd4-491d-a0ce-51519a35486d" containerID="cb92081d44e779c1995c55501d5f98901ddd3f07f0de92e04c6baceafaf829f9" exitCode=0
Oct 30 05:31:36.459070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.459067    5066 generic.go:356] "Generic (PLEG): container finished" podID="e71a9464-fdd4-491d-a0ce-51519a35486d" containerID="2f25d09378ce9139562b573776a154398a698ac217207a64e98d121205465b29" exitCode=0
Oct 30 05:31:36.459265 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.459075    5066 generic.go:356] "Generic (PLEG): container finished" podID="e71a9464-fdd4-491d-a0ce-51519a35486d" containerID="25c29d007fd1a4b08bc1a56919b26d380dffcc03af69c2fa1295b0d4f9bee077" exitCode=0
Oct 30 05:31:36.459265 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.459087    5066 generic.go:356] "Generic (PLEG): container finished" podID="e71a9464-fdd4-491d-a0ce-51519a35486d" containerID="5dc76552cfcf53c013fc56ed27faaded265ad672f530d660407c1cad9981ccb5" exitCode=0
Oct 30 05:31:36.459265 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.459095    5066 generic.go:356] "Generic (PLEG): container finished" podID="e71a9464-fdd4-491d-a0ce-51519a35486d" containerID="e681a8157306c319fad1feda6ad19b73845bd932e4132f6d7a20cd759699db22" exitCode=0
Oct 30 05:31:36.459265 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.459103    5066 generic.go:356] "Generic (PLEG): container finished" podID="e71a9464-fdd4-491d-a0ce-51519a35486d" containerID="44b23df4824142e19d11d9220314bf4a4b039586b9488ea42c1bbd7dbe4ced0e" exitCode=0
Oct 30 05:31:36.473395 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.473377    5066 manager.go:324] Recovery completed
Oct 30 05:31:36.494385 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:36.494354    5066 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 05:31:36.495452 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495436    5066 cpu_manager.go:226] "Starting" policy="none"
Oct 30 05:31:36.495452 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495450    5066 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 05:31:36.495553 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495465    5066 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 05:31:36.495603 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495578    5066 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 05:31:36.495635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495606    5066 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 05:31:36.495635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495624    5066 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 05:31:36.495635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495630    5066 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 05:31:36.495635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495634    5066 policy_none.go:50] "Start"
Oct 30 05:31:36.495764 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495643    5066 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 05:31:36.495764 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495652    5066 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 05:31:36.495817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495774    5066 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 05:31:36.495817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495780    5066 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 05:31:36.495817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.495786    5066 policy_none.go:44] "Start"
Oct 30 05:31:36.500270 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.500243    5066 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 05:31:36.502790 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.502776    5066 manager.go:341] "Starting Device Plugin manager"
Oct 30 05:31:36.503341 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:36.503322    5066 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 05:31:36.503341 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.503340    5066 server.go:87] "Starting device plugin registration server"
Oct 30 05:31:36.503631 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.503619    5066 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 05:31:36.503705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.503632    5066 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 05:31:36.503786 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.503773    5066 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 05:31:36.503894 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.503861    5066 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 05:31:36.504046 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.503910    5066 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 05:31:36.504046 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.503917    5066 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 05:31:36.504046 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.503965    5066 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T05:31:36.503889324Z","UUID":"530cb156-6a6d-447a-9eba-28be4e63b49a","Handler":null,"Name":"","Endpoint":""}
Oct 30 05:31:36.506384 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:36.505271    5066 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 05:31:36.506384 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.505733    5066 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 05:31:36.506384 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.505761    5066 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 05:31:36.604767 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.604731    5066 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 05:31:36.606251 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.606230    5066 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 05:31:36.606376 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.606261    5066 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 05:31:36.606376 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.606271    5066 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-173.ec2.internal" event="NodeHasSufficientPID"
Oct 30 05:31:36.606376 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.606315    5066 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.645341 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.645314    5066 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.645458 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.645391    5066 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.694569 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.694466    5066 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"]
Oct 30 05:31:36.695196 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.695155    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal" event={"ID":"6f17565975b3f18442fbd67650b5b56f","Type":"ContainerStarted","Data":"8cab33c9540f3a3ee6bf3594699fd4402bf5cab06448e0e6fa441d0beb2b6463"}
Oct 30 05:31:36.695196 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.695192    5066 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.695278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.695210    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal" event={"ID":"6f17565975b3f18442fbd67650b5b56f","Type":"ContainerDied","Data":"b5cbaa1666f97cc808001b56e15afbb07cf03552c94130d44e6d4c321f85f74d"}
Oct 30 05:31:36.695278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.695230    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal" event={"ID":"6f17565975b3f18442fbd67650b5b56f","Type":"ContainerStarted","Data":"81b6c9eea343a12127abee87744a659b4bbfc749aa6ac9f7ced7926d54a222bb"}
Oct 30 05:31:36.695278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.695248    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal" event={"ID":"ee22bb4b0b2b5043024e8d4645898710","Type":"ContainerStarted","Data":"e508dc326c3dd40a245acbd04d22c1afc57adec21e74f0fa0127b638b45e6802"}
Oct 30 05:31:36.695278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.695263    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal" event={"ID":"ee22bb4b0b2b5043024e8d4645898710","Type":"ContainerStarted","Data":"ee0e9dd0e3fbb8cec8916714da03b17114aa8711de5614358f41153935f4eeef"}
Oct 30 05:31:36.695471 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.695453    5066 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.722447 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.722416    5066 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 05:31:36.722545 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:36.722466    5066 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-173.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.723039 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.723023    5066 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 05:31:36.723119 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:36.723070    5066 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.822380 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.822344    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee22bb4b0b2b5043024e8d4645898710-config\") pod \"kube-apiserver-proxy-ip-10-0-136-173.ec2.internal\" (UID: \"ee22bb4b0b2b5043024e8d4645898710\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.822524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.822385    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6f17565975b3f18442fbd67650b5b56f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" (UID: \"6f17565975b3f18442fbd67650b5b56f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.822524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.822411    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f17565975b3f18442fbd67650b5b56f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" (UID: \"6f17565975b3f18442fbd67650b5b56f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.923478 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.923443    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee22bb4b0b2b5043024e8d4645898710-config\") pod \"kube-apiserver-proxy-ip-10-0-136-173.ec2.internal\" (UID: \"ee22bb4b0b2b5043024e8d4645898710\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.923478 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.923477    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6f17565975b3f18442fbd67650b5b56f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" (UID: \"6f17565975b3f18442fbd67650b5b56f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.923698 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.923507    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f17565975b3f18442fbd67650b5b56f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" (UID: \"6f17565975b3f18442fbd67650b5b56f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.923698 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.923551    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f17565975b3f18442fbd67650b5b56f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" (UID: \"6f17565975b3f18442fbd67650b5b56f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.923698 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.923568    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6f17565975b3f18442fbd67650b5b56f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" (UID: \"6f17565975b3f18442fbd67650b5b56f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:36.923698 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:36.923551    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ee22bb4b0b2b5043024e8d4645898710-config\") pod \"kube-apiserver-proxy-ip-10-0-136-173.ec2.internal\" (UID: \"ee22bb4b0b2b5043024e8d4645898710\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:37.350271 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.350240    5066 apiserver.go:52] "Watching apiserver"
Oct 30 05:31:37.384566 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.384539    5066 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 05:31:37.385338 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.385303    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal","openshift-multus/multus-additional-cni-plugins-g6l5g","openshift-network-diagnostics/network-check-target-fdfg7","openshift-dns/dns-default-bzbfz","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb","openshift-insights/insights-operator-7c4986d449-swtbv","openshift-multus/network-metrics-daemon-pfdj6","kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal","openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf","openshift-insights/periodic-gathering-4pjwt-q72n8","openshift-ovn-kubernetes/ovnkube-node-hpgtg","openshift-ingress-canary/ingress-canary-mfzzm","kube-system/global-pull-secret-syncer-kkcc6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897","openshift-insights/insights-runtime-extractor-6864g","openshift-network-operator/iptables-alerter-fvfm6","kube-system/kas-connection-checker-685895c7c9-4sd8r","openshift-console-operator/console-operator-7cd56b8dbc-58dpm","openshift-image-registry/image-registry-6b668646-vshtx","openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9","kube-system/kas-connection-checker-685895c7c9-gh627","openshift-image-registry/node-ca-qw6mh","openshift-multus/multus-cbf62","openshift-network-console/networking-console-plugin-86c478bb88-v6b7k","openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd","openshift-ingress/router-default-59857694c5-wq9fj","kube-system/kas-connection-checker-685895c7c9-k5d59","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq","openshift-dns/node-resolver-mtj5z","openshift-image-registry/image-registry-7d65554659-bdvbl","kube-system/konnectivity-agent-bfpdf","openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts","openshift-cluster-node-tuning-operator/tuned-9gcnl","openshift-service-ca/service-ca-5fffc97bdf-6xwxq","openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"]
Oct 30 05:31:37.387154 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.387135    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.388837 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.388236    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:37.391911 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.391886    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:37.392266 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.392253    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:37.393298 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.393282    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:37.393946 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.393928    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:37.395566 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.395549    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.399719 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.399705    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:37.400802 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.400790    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.414723 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.414696    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 05:31:37.415040 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.415022    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.415241 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.415227    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 05:31:37.422088 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.422063    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 05:31:37.422982 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.422957    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 05:31:37.423079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423015    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 05:31:37.423079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423029    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-st5hg\""
Oct 30 05:31:37.423186 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423095    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 05:31:37.423219 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423185    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bmrwn\""
Oct 30 05:31:37.423402 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423383    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 05:31:37.423468 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423437    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xpx6p\""
Oct 30 05:31:37.423468 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423459    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 05:31:37.423555 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423522    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 05:31:37.423615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423567    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 05:31:37.423686 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423660    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 05:31:37.423774 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423760    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jvn8d\""
Oct 30 05:31:37.423830 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423774    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 05:31:37.423830 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423808    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-nr89r\""
Oct 30 05:31:37.423915 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423850    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 05:31:37.423915 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.423861    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 05:31:37.424247 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.424222    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qrqvm\""
Oct 30 05:31:37.424338 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.424326    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mjhr7\""
Oct 30 05:31:37.425636 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.425614    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lv7bs\""
Oct 30 05:31:37.425858 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.425835    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 05:31:37.425937 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.425899    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jtd4z\""
Oct 30 05:31:37.426127 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.426109    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-7pqs8\""
Oct 30 05:31:37.426339 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.426323    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-2ztbt\""
Oct 30 05:31:37.426458 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.426437    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v8jj4\""
Oct 30 05:31:37.426710 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.426682    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 05:31:37.426807 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.426778    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gf7fl\""
Oct 30 05:31:37.427065 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.427047    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f8j26\""
Oct 30 05:31:37.427427 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.427402    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2msqk\""
Oct 30 05:31:37.427941 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.427917    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-cxh97\""
Oct 30 05:31:37.428029 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.427999    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 05:31:37.428172 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.428153    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 05:31:37.428236 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.428212    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 05:31:37.428283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.428265    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 05:31:37.428458 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.428443    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7pqn7\""
Oct 30 05:31:37.428668 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.428632    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 05:31:37.428754 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.428721    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wks87\""
Oct 30 05:31:37.428834 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.428510    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 05:31:37.428923 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.428894    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-vg6rp\""
Oct 30 05:31:37.429174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.429154    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 05:31:37.429238 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.429226    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-k22sc\""
Oct 30 05:31:37.429951 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.429930    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.430207 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.430184    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.430715 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.430679    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.430804 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.430733    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 05:31:37.430804 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.430777    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.430904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.430838    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 05:31:37.430967 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.430917    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.431047 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.430953    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.431047 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.430990    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 05:31:37.431187 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.431173    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 05:31:37.431219 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.431182    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.431219 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.431194    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.431751 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.431736    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.431809 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.431750    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.431809 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.430618    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.431809 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.431775    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 05:31:37.431947 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.431808    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.431947 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.431838    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.431947 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.431880    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.431947 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.431912    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.432102 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.431984    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.432102 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432042    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 05:31:37.432102 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432048    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.432102 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432101    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 05:31:37.432300 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432115    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.432300 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432132    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 05:31:37.432300 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432180    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 05:31:37.432300 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432190    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 05:31:37.432300 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432222    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 05:31:37.432300 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432226    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.432300 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432262    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 05:31:37.432627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432308    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 05:31:37.432627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432337    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.432627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432340    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.432627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432354    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.432627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432405    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.432627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432408    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 05:31:37.432627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432450    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.432627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432457    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.432627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432470    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.432627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432551    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 05:31:37.432627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432564    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.432627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432582    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.432964 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432644    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 05:31:37.432964 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432699    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.432964 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.432746    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.433618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.433605    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.439526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.439509    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 05:31:37.439618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.439558    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 05:31:37.439618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.439570    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.439716 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.439633    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.439716 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.439649    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.439716 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.439676    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 05:31:37.439716 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.439684    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.439888 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.439812    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 05:31:37.439888 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.439841    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.439888 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.439863    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.440003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.439992    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 05:31:37.440079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.440065    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 05:31:37.440159 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.440145    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 05:31:37.440374 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.440360    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 05:31:37.441157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.441142    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 05:31:37.463226 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.463206    5066 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:37.463377 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.463343    5066 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:37.466242 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.466226    5066 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 05:31:37.488538 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.488517    5066 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 05:31:37.488641 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:37.488562    5066 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-173.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:37.488910 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.488896    5066 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 05:31:37.488956 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:37.488927    5066 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-173.ec2.internal"
Oct 30 05:31:37.527571 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.527543    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-image-registry-private-configuration\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.527571 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.527573    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7319282a-25bd-402e-a55c-8cc81da483c1-agent-certs\") pod \"konnectivity-agent-bfpdf\" (UID: \"7319282a-25bd-402e-a55c-8cc81da483c1\") " pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:31:37.527825 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.527615    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysconfig\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.527825 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.527745    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-systemd\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.527825 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.527778    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.527825 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.527813    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:37.527998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.527862    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-image-registry-private-configuration\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.527998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.527880    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7319282a-25bd-402e-a55c-8cc81da483c1-agent-certs\") pod \"konnectivity-agent-bfpdf\" (UID: \"7319282a-25bd-402e-a55c-8cc81da483c1\") " pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:31:37.527998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.527928    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvw56\" (UniqueName: \"kubernetes.io/projected/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-api-access-cvw56\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.527998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.527957    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/86fadfbe-f000-4b42-a8ff-7b67061959c5-snapshots\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:37.528174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528031    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-ovn\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.528174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528065    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/86fadfbe-f000-4b42-a8ff-7b67061959c5-snapshots\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:37.528174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528116    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-var-lib-kubelet\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.528174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528126    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.528174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528163    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l9nz\" (UniqueName: \"kubernetes.io/projected/06639c1a-71fb-47d6-898f-464546d8b66a-kube-api-access-9l9nz\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:37.528392 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528182    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-systemd\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.528392 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528216    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-bound-sa-token\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.528392 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528283    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/06639c1a-71fb-47d6-898f-464546d8b66a-signing-cabundle\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:37.528392 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528385    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.528581 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528423    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m7s7\" (UniqueName: \"kubernetes.io/projected/fcdb9d1a-474b-49f3-8683-f3874ad9d769-kube-api-access-5m7s7\") pod \"kas-connection-checker-685895c7c9-gh627\" (UID: \"fcdb9d1a-474b-49f3-8683-f3874ad9d769\") " pod="kube-system/kas-connection-checker-685895c7c9-gh627"
Oct 30 05:31:37.528581 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528459    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rznfw\" (UniqueName: \"kubernetes.io/projected/02a30d07-33cc-4be0-a920-8e839f8a493b-kube-api-access-rznfw\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.528581 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528532    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-installation-pull-secrets\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.528581 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528541    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/06639c1a-71fb-47d6-898f-464546d8b66a-signing-cabundle\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:37.528581 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528564    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-crio-socket\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.528843 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528625    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-k8s-cni-cncf-io\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.528843 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528664    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-multus-certs\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.528843 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528730    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-certificates\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.528843 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528769    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.528843 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528800    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41edacc9-dea1-4343-96f5-5caf8a82fe84-serving-cert\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:37.528843 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528827    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-conf-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.528843 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528771    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-installation-pull-secrets\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.529188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528852    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:37.529188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528882    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdbt\" (UniqueName: \"kubernetes.io/projected/c46db00b-3c48-4d8d-844c-918d8126cc41-kube-api-access-7xdbt\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:37.529188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528905    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-certificates\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.529188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528909    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.529188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.528972    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41edacc9-dea1-4343-96f5-5caf8a82fe84-config\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:37.529188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529001    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-trusted-ca\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.529188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529023    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-socket-dir-parent\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.529188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529039    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-daemon-config\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.529188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529055    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0cf86020-761f-4fbc-808e-7dc6c5d5515f-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:37.529188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529071    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-kubelet-config\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:37.529188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529154    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41edacc9-dea1-4343-96f5-5caf8a82fe84-config\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529196    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41edacc9-dea1-4343-96f5-5caf8a82fe84-serving-cert\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529213    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9hln\" (UniqueName: \"kubernetes.io/projected/834a0562-2972-449c-9ec4-a69908732c37-kube-api-access-z9hln\") pod \"migrator-66df5d496b-rq4pd\" (UID: \"834a0562-2972-449c-9ec4-a69908732c37\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529258    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dffbj\" (UniqueName: \"kubernetes.io/projected/4684e0e6-736f-41bf-8f22-77245e339d58-kube-api-access-dffbj\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529260    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0cf86020-761f-4fbc-808e-7dc6c5d5515f-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529304    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-cni-bin\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529337    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrgd8\" (UniqueName: \"kubernetes.io/projected/1c2414f4-fa94-4054-9b41-b264e6088e67-kube-api-access-mrgd8\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529354    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-daemon-config\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529387    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-nginx-conf\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529398    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-trusted-ca\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529424    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529450    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529483    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-system-cni-dir\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529526    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7pkr\" (UniqueName: \"kubernetes.io/projected/666a74c6-3979-480a-b4d7-4872c84b2413-kube-api-access-n7pkr\") pod \"kas-connection-checker-685895c7c9-k5d59\" (UID: \"666a74c6-3979-480a-b4d7-4872c84b2413\") " pod="kube-system/kas-connection-checker-685895c7c9-k5d59"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529560    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d4vl\" (UniqueName: \"kubernetes.io/projected/dfbf26d6-b6c8-4d33-9219-5e9fff25a103-kube-api-access-8d4vl\") pod \"network-check-source-6f78d59f99-dvtwf\" (UID: \"dfbf26d6-b6c8-4d33-9219-5e9fff25a103\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529578    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-nginx-conf\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529628    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6b6c8b1-dea3-45a1-b78f-676de9310b70-iptables-alerter-script\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:31:37.529718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529674    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529755    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-cnibin\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529796    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tmzj\" (UniqueName: \"kubernetes.io/projected/41edacc9-dea1-4343-96f5-5caf8a82fe84-kube-api-access-9tmzj\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529826    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-socket-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529876    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a6b6c8b1-dea3-45a1-b78f-676de9310b70-iptables-alerter-script\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529861    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-sys-fs\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529895    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-original-pull-secret\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.529914    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530011    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c2414f4-fa94-4054-9b41-b264e6088e67-cni-binary-copy\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530032    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf86020-761f-4fbc-808e-7dc6c5d5515f-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530051    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l72vm\" (UniqueName: \"kubernetes.io/projected/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-kube-api-access-l72vm\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530068    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq7d9\" (UniqueName: \"kubernetes.io/projected/a655cfec-4ee3-4ac9-badc-4fb318603506-kube-api-access-pq7d9\") pod \"ingress-canary-mfzzm\" (UID: \"a655cfec-4ee3-4ac9-badc-4fb318603506\") " pod="openshift-ingress-canary/ingress-canary-mfzzm"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530091    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-system-cni-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530097    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530107    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-run-ovn-kubernetes\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530126    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-cni-binary-copy\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530237    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c2414f4-fa94-4054-9b41-b264e6088e67-cni-binary-copy\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530243    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86fadfbe-f000-4b42-a8ff-7b67061959c5-serving-cert\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530277    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-bound-sa-token\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530296    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e71a9464-fdd4-491d-a0ce-51519a35486d-cni-binary-copy\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530324    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-host\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530344    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-registration-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530358    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530380    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4c2b487-ec63-4b07-a11c-781cb9afd37d-hosts-file\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530414    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-data-volume\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.530436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530452    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwdkg\" (UniqueName: \"kubernetes.io/projected/abe28d56-457c-4e87-9cb1-804247d48da0-kube-api-access-dwdkg\") pod \"migrator-66df5d496b-n6zg9\" (UID: \"abe28d56-457c-4e87-9cb1-804247d48da0\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530476    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf86020-761f-4fbc-808e-7dc6c5d5515f-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530517    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-data-volume\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530562    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab824d2b-de9a-4384-a4a8-166f58808be8-serving-cert\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530611    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-trusted-ca-bundle\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530641    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86fadfbe-f000-4b42-a8ff-7b67061959c5-serving-cert\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530698    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrchp\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-kube-api-access-wrchp\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530784    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530796    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ab824d2b-de9a-4384-a4a8-166f58808be8-serving-cert\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530870    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-default-certificate\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530903    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44rjf\" (UniqueName: \"kubernetes.io/projected/60bf362c-030f-4adf-9878-842b2fdc326b-kube-api-access-44rjf\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530904    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-trusted-ca-bundle\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.530933    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-run-netns\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531003    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-node-log\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531084    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-default-certificate\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531084    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-ovnkube-script-lib\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.531161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531116    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-kubernetes\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531242    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-tuned\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531262    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c46db00b-3c48-4d8d-844c-918d8126cc41-config-volume\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531279    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-var-lib-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531293    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-ovnkube-config\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531312    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c46db00b-3c48-4d8d-844c-918d8126cc41-tmp-dir\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531336    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531339    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-ovnkube-script-lib\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531375    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-tuned\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531391    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c46db00b-3c48-4d8d-844c-918d8126cc41-tmp-dir\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531380    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djp28\" (UniqueName: \"kubernetes.io/projected/e71a9464-fdd4-491d-a0ce-51519a35486d-kube-api-access-djp28\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531444    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4c2b487-ec63-4b07-a11c-781cb9afd37d-tmp-dir\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531481    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct475\" (UniqueName: \"kubernetes.io/projected/a0ea8679-d95b-49d6-a614-9bb4eb4ae029-kube-api-access-ct475\") pod \"kas-connection-checker-685895c7c9-4sd8r\" (UID: \"a0ea8679-d95b-49d6-a614-9bb4eb4ae029\") " pod="kube-system/kas-connection-checker-685895c7c9-4sd8r"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531482    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c46db00b-3c48-4d8d-844c-918d8126cc41-config-volume\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531511    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-ovnkube-config\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531529    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4c2b487-ec63-4b07-a11c-781cb9afd37d-tmp-dir\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531616    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab824d2b-de9a-4384-a4a8-166f58808be8-config\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531636    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-certificates\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531651    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96hw6\" (UniqueName: \"kubernetes.io/projected/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-kube-api-access-96hw6\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531668    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-netns\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.531705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531692    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-hostroot\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531789    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531801    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-certificates\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531812    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-stats-auth\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531837    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531879    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7319282a-25bd-402e-a55c-8cc81da483c1-konnectivity-ca\") pod \"konnectivity-agent-bfpdf\" (UID: \"7319282a-25bd-402e-a55c-8cc81da483c1\") " pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531898    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ab824d2b-de9a-4384-a4a8-166f58808be8-config\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531955    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-ca-trust-extracted\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531987    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-cni-bin\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.531993    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-stats-auth\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532007    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7319282a-25bd-402e-a55c-8cc81da483c1-konnectivity-ca\") pod \"konnectivity-agent-bfpdf\" (UID: \"7319282a-25bd-402e-a55c-8cc81da483c1\") " pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532013    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-cni-multus\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532039    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dac2ba5d-8990-47e3-aa65-02ff7500ac01-ca-trust-extracted\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532059    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-ca-trust-extracted\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532064    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysctl-d\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532089    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wkpm\" (UniqueName: \"kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm\") pod \"network-check-target-fdfg7\" (UID: \"f82afdb2-9cea-479b-8d0b-88a814886c3a\") " pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532105    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dac2ba5d-8990-47e3-aa65-02ff7500ac01-ca-trust-extracted\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532228    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86fadfbe-f000-4b42-a8ff-7b67061959c5-tmp\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532247    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml8s5\" (UniqueName: \"kubernetes.io/projected/2f92fa2c-9133-484f-b326-a12b2b618900-kube-api-access-ml8s5\") pod \"volume-data-source-validator-58c9f5f46d-gshpq\" (UID: \"2f92fa2c-9133-484f-b326-a12b2b618900\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532263    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxnt\" (UniqueName: \"kubernetes.io/projected/f74041a6-7409-439c-9ecb-4bc8dcc2006e-kube-api-access-rfxnt\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532277    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-trusted-ca\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532294    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77k6x\" (UniqueName: \"kubernetes.io/projected/b4c2b487-ec63-4b07-a11c-781cb9afd37d-kube-api-access-77k6x\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532319    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-kubelet\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.532470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532371    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86fadfbe-f000-4b42-a8ff-7b67061959c5-tmp\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:37.533711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532692    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv985\" (UniqueName: \"kubernetes.io/projected/0cf86020-761f-4fbc-808e-7dc6c5d5515f-kube-api-access-sv985\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:37.533711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532727    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86fadfbe-f000-4b42-a8ff-7b67061959c5-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:37.533711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532759    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:37.533711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.532802    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-service-ca-bundle\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:37.533711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.533298    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysctl-conf\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.533711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.533358    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.533711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.533403    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-slash\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.533711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.533435    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.533711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.533503    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bptqj\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-kube-api-access-bptqj\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.533711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.533536    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-cnibin\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.533711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.533577    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6b6c8b1-dea3-45a1-b78f-676de9310b70-host-slash\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:31:37.534212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.533830    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-service-ca-bundle\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:37.534212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.533739    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:37.534212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.533905    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:37.534212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.533983    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-trusted-ca\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.534212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534046    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02a30d07-33cc-4be0-a920-8e839f8a493b-ovn-node-metrics-cert\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.534212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534198    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-serviceca\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:31:37.534469 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534228    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-host\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.534469 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534276    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-image-registry-private-configuration\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.534469 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534346    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a655cfec-4ee3-4ac9-badc-4fb318603506-cert\") pod \"ingress-canary-mfzzm\" (UID: \"a655cfec-4ee3-4ac9-badc-4fb318603506\") " pod="openshift-ingress-canary/ingress-canary-mfzzm"
Oct 30 05:31:37.534469 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534370    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02a30d07-33cc-4be0-a920-8e839f8a493b-ovn-node-metrics-cert\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.534469 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534383    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86fadfbe-f000-4b42-a8ff-7b67061959c5-service-ca-bundle\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:37.534469 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534430    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-log-socket\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.534773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534488    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-cni-netd\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.534773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534510    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-serviceca\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:31:37.534773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534530    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-sys\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.534773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534542    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86fadfbe-f000-4b42-a8ff-7b67061959c5-service-ca-bundle\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:37.534773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534555    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab824d2b-de9a-4384-a4a8-166f58808be8-trusted-ca\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:37.534773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534576    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-etc-kubernetes\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.534773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534725    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-image-registry-private-configuration\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.534773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534742    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/f74041a6-7409-439c-9ecb-4bc8dcc2006e-archives-path\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:37.534773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534766    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-kubelet\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.535168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534865    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a655cfec-4ee3-4ac9-badc-4fb318603506-cert\") pod \"ingress-canary-mfzzm\" (UID: \"a655cfec-4ee3-4ac9-badc-4fb318603506\") " pod="openshift-ingress-canary/ingress-canary-mfzzm"
Oct 30 05:31:37.535168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534886    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-etc-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.535168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534900    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/f74041a6-7409-439c-9ecb-4bc8dcc2006e-archives-path\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:37.535168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534925    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-modprobe-d\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.535168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534961    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-device-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.535168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.534986    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86fadfbe-f000-4b42-a8ff-7b67061959c5-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:37.535168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535046    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-run\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.535168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535083    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-lib-modules\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.535168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535144    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-cni-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.535538 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535194    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-env-overrides\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.535538 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535221    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-installation-pull-secrets\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.535538 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535312    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7cjr\" (UniqueName: \"kubernetes.io/projected/0421f010-8e05-4896-b171-301e85964bb5-kube-api-access-s7cjr\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.535538 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535345    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvb6d\" (UniqueName: \"kubernetes.io/projected/46c256fd-a89b-41f9-95aa-d8e81f783820-kube-api-access-rvb6d\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:37.535538 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535357    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab824d2b-de9a-4384-a4a8-166f58808be8-trusted-ca\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:37.535538 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535372    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.535538 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535495    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/06639c1a-71fb-47d6-898f-464546d8b66a-signing-key\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:37.535538 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535532    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdz8s\" (UniqueName: \"kubernetes.io/projected/ab824d2b-de9a-4384-a4a8-166f58808be8-kube-api-access-xdz8s\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:37.535950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535566    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfzsh\" (UniqueName: \"kubernetes.io/projected/86fadfbe-f000-4b42-a8ff-7b67061959c5-kube-api-access-jfzsh\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:37.535950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535531    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02a30d07-33cc-4be0-a920-8e839f8a493b-env-overrides\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.535950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535798    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zcgx\" (UniqueName: \"kubernetes.io/projected/a6b6c8b1-dea3-45a1-b78f-676de9310b70-kube-api-access-8zcgx\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:31:37.535950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535834    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-systemd-units\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.535950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535866    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.536171 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.535993    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-installation-pull-secrets\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.536171 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.536088    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rd8b\" (UniqueName: \"kubernetes.io/projected/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-kube-api-access-8rd8b\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:31:37.536171 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.536122    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-tmp\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.536171 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.536149    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-dbus\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:37.536340 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.536235    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.536340 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.536308    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/06639c1a-71fb-47d6-898f-464546d8b66a-signing-key\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:37.536877 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.536498    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-tmp\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.552862 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.552842    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dffbj\" (UniqueName: \"kubernetes.io/projected/4684e0e6-736f-41bf-8f22-77245e339d58-kube-api-access-dffbj\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:37.553168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.553145    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-bound-sa-token\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.553208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.553191    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rznfw\" (UniqueName: \"kubernetes.io/projected/02a30d07-33cc-4be0-a920-8e839f8a493b-kube-api-access-rznfw\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.553407 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.553392    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrgd8\" (UniqueName: \"kubernetes.io/projected/1c2414f4-fa94-4054-9b41-b264e6088e67-kube-api-access-mrgd8\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.554544 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.554521    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7pkr\" (UniqueName: \"kubernetes.io/projected/666a74c6-3979-480a-b4d7-4872c84b2413-kube-api-access-n7pkr\") pod \"kas-connection-checker-685895c7c9-k5d59\" (UID: \"666a74c6-3979-480a-b4d7-4872c84b2413\") " pod="kube-system/kas-connection-checker-685895c7c9-k5d59"
Oct 30 05:31:37.554584 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.554574    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xdbt\" (UniqueName: \"kubernetes.io/projected/c46db00b-3c48-4d8d-844c-918d8126cc41-kube-api-access-7xdbt\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:37.555144 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.555124    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m7s7\" (UniqueName: \"kubernetes.io/projected/fcdb9d1a-474b-49f3-8683-f3874ad9d769-kube-api-access-5m7s7\") pod \"kas-connection-checker-685895c7c9-gh627\" (UID: \"fcdb9d1a-474b-49f3-8683-f3874ad9d769\") " pod="kube-system/kas-connection-checker-685895c7c9-gh627"
Oct 30 05:31:37.555216 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.555172    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvw56\" (UniqueName: \"kubernetes.io/projected/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-api-access-cvw56\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.555338 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.555322    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l9nz\" (UniqueName: \"kubernetes.io/projected/06639c1a-71fb-47d6-898f-464546d8b66a-kube-api-access-9l9nz\") pod \"service-ca-5fffc97bdf-6xwxq\" (UID: \"06639c1a-71fb-47d6-898f-464546d8b66a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-6xwxq"
Oct 30 05:31:37.560015 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.559997    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9hln\" (UniqueName: \"kubernetes.io/projected/834a0562-2972-449c-9ec4-a69908732c37-kube-api-access-z9hln\") pod \"migrator-66df5d496b-rq4pd\" (UID: \"834a0562-2972-449c-9ec4-a69908732c37\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-rq4pd"
Oct 30 05:31:37.560487 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.560472    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44rjf\" (UniqueName: \"kubernetes.io/projected/60bf362c-030f-4adf-9878-842b2fdc326b-kube-api-access-44rjf\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:37.561171 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.561154    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv985\" (UniqueName: \"kubernetes.io/projected/0cf86020-761f-4fbc-808e-7dc6c5d5515f-kube-api-access-sv985\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-j55lb\" (UID: \"0cf86020-761f-4fbc-808e-7dc6c5d5515f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb"
Oct 30 05:31:37.561649 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.561630    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77k6x\" (UniqueName: \"kubernetes.io/projected/b4c2b487-ec63-4b07-a11c-781cb9afd37d-kube-api-access-77k6x\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:31:37.561757 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.561744    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml8s5\" (UniqueName: \"kubernetes.io/projected/2f92fa2c-9133-484f-b326-a12b2b618900-kube-api-access-ml8s5\") pod \"volume-data-source-validator-58c9f5f46d-gshpq\" (UID: \"2f92fa2c-9133-484f-b326-a12b2b618900\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-gshpq"
Oct 30 05:31:37.561823 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.561809    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bptqj\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-kube-api-access-bptqj\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.561919 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.561904    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfxnt\" (UniqueName: \"kubernetes.io/projected/f74041a6-7409-439c-9ecb-4bc8dcc2006e-kube-api-access-rfxnt\") pod \"periodic-gathering-4pjwt-q72n8\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") " pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:31:37.569633 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.569584    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d4vl\" (UniqueName: \"kubernetes.io/projected/dfbf26d6-b6c8-4d33-9219-5e9fff25a103-kube-api-access-8d4vl\") pod \"network-check-source-6f78d59f99-dvtwf\" (UID: \"dfbf26d6-b6c8-4d33-9219-5e9fff25a103\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-dvtwf"
Oct 30 05:31:37.569758 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.569731    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvb6d\" (UniqueName: \"kubernetes.io/projected/46c256fd-a89b-41f9-95aa-d8e81f783820-kube-api-access-rvb6d\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:37.569848 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.569828    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfzsh\" (UniqueName: \"kubernetes.io/projected/86fadfbe-f000-4b42-a8ff-7b67061959c5-kube-api-access-jfzsh\") pod \"insights-operator-7c4986d449-swtbv\" (UID: \"86fadfbe-f000-4b42-a8ff-7b67061959c5\") " pod="openshift-insights/insights-operator-7c4986d449-swtbv"
Oct 30 05:31:37.569899 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.569883    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwdkg\" (UniqueName: \"kubernetes.io/projected/abe28d56-457c-4e87-9cb1-804247d48da0-kube-api-access-dwdkg\") pod \"migrator-66df5d496b-n6zg9\" (UID: \"abe28d56-457c-4e87-9cb1-804247d48da0\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-n6zg9"
Oct 30 05:31:37.570107 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.570074    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96hw6\" (UniqueName: \"kubernetes.io/projected/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-kube-api-access-96hw6\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.570620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.570557    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zcgx\" (UniqueName: \"kubernetes.io/projected/a6b6c8b1-dea3-45a1-b78f-676de9310b70-kube-api-access-8zcgx\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:31:37.570763 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.570741    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wkpm\" (UniqueName: \"kubernetes.io/projected/f82afdb2-9cea-479b-8d0b-88a814886c3a-kube-api-access-5wkpm\") pod \"network-check-target-fdfg7\" (UID: \"f82afdb2-9cea-479b-8d0b-88a814886c3a\") " pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:37.570834 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.570759    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l72vm\" (UniqueName: \"kubernetes.io/projected/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-kube-api-access-l72vm\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:37.570894 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.570859    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrchp\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-kube-api-access-wrchp\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.570954 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.570924    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdz8s\" (UniqueName: \"kubernetes.io/projected/ab824d2b-de9a-4384-a4a8-166f58808be8-kube-api-access-xdz8s\") pod \"console-operator-7cd56b8dbc-58dpm\" (UID: \"ab824d2b-de9a-4384-a4a8-166f58808be8\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:37.571053 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.571036    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7cjr\" (UniqueName: \"kubernetes.io/projected/0421f010-8e05-4896-b171-301e85964bb5-kube-api-access-s7cjr\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.571125 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.571110    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tmzj\" (UniqueName: \"kubernetes.io/projected/41edacc9-dea1-4343-96f5-5caf8a82fe84-kube-api-access-9tmzj\") pod \"service-ca-operator-544d749c8c-q2fx2\" (UID: \"41edacc9-dea1-4343-96f5-5caf8a82fe84\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2"
Oct 30 05:31:37.572022 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.572002    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct475\" (UniqueName: \"kubernetes.io/projected/a0ea8679-d95b-49d6-a614-9bb4eb4ae029-kube-api-access-ct475\") pod \"kas-connection-checker-685895c7c9-4sd8r\" (UID: \"a0ea8679-d95b-49d6-a614-9bb4eb4ae029\") " pod="kube-system/kas-connection-checker-685895c7c9-4sd8r"
Oct 30 05:31:37.572505 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.572488    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rd8b\" (UniqueName: \"kubernetes.io/projected/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-kube-api-access-8rd8b\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:31:37.574230 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.574213    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djp28\" (UniqueName: \"kubernetes.io/projected/e71a9464-fdd4-491d-a0ce-51519a35486d-kube-api-access-djp28\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.574622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.574603    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq7d9\" (UniqueName: \"kubernetes.io/projected/a655cfec-4ee3-4ac9-badc-4fb318603506-kube-api-access-pq7d9\") pod \"ingress-canary-mfzzm\" (UID: \"a655cfec-4ee3-4ac9-badc-4fb318603506\") " pod="openshift-ingress-canary/ingress-canary-mfzzm"
Oct 30 05:31:37.575291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.575272    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-bound-sa-token\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.637085 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637003    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysctl-conf\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.637085 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637038    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.637085 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637060    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-slash\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.637085 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637083    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637116    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-cnibin\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637154    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6b6c8b1-dea3-45a1-b78f-676de9310b70-host-slash\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637172    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-cnibin\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637188    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637215    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-host\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637218    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637217    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysctl-conf\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637158    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-slash\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637234    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-log-socket\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637239    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6b6c8b1-dea3-45a1-b78f-676de9310b70-host-slash\") pod \"iptables-alerter-fvfm6\" (UID: \"a6b6c8b1-dea3-45a1-b78f-676de9310b70\") " pod="openshift-network-operator/iptables-alerter-fvfm6"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637251    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-cni-netd\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637274    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-host\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637282    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-cni-netd\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637288    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-sys\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637309    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-log-socket\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637316    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-etc-kubernetes\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637341    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-kubelet\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637352    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-sys\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.637353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637364    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-etc-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637389    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-kubelet\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637392    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-modprobe-d\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637424    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-device-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637445    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-run\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637466    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-lib-modules\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637402    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-etc-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637400    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-etc-kubernetes\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637492    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-cni-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637499    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-modprobe-d\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637497    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-device-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637522    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637525    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-run\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637535    5066 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637548    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-systemd-units\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637567    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-cni-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637572    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637612    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637623    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-systemd-units\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637626    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-lib-modules\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637629    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-dbus\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637669    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637690    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637680    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-dbus\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637712    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637736    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637768    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysconfig\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637806    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-systemd\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637835    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637863    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-ovn\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637865    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-systemd\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637864    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysconfig\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637895    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-var-lib-kubelet\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637911    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-ovn\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637921    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-systemd\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637947    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-var-lib-kubelet\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:37.637951    5066 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:37.638004    5066 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.637946    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:37.638036    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls podName:bfa3e142-50e3-4b85-83fa-46f7dc8b7858 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:38.138014398 +0000 UTC m=+1.883299733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6xnts" (UID: "bfa3e142-50e3-4b85-83fa-46f7dc8b7858") : secret "cluster-monitoring-operator-tls" not found
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:37.638054    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls podName:ecfec034-dabf-45fb-a6cb-ef3ace810cb2 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:38.138044298 +0000 UTC m=+1.883329628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6864g" (UID: "ecfec034-dabf-45fb-a6cb-ef3ace810cb2") : secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638078    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-crio-socket\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638101    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-k8s-cni-cncf-io\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638123    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-multus-certs\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638138    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-crio-socket\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638147    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-conf-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638128    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-run-systemd\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638172    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:37.638139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638179    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-k8s-cni-cncf-io\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638197    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-socket-dir-parent\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638205    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-conf-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638200    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-multus-certs\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638222    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-kubelet-config\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638242    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-multus-socket-dir-parent\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638249    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-cni-bin\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:37.638270    5066 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638278    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638292    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa-kubelet-config\") pod \"global-pull-secret-syncer-kkcc6\" (UID: \"7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa\") " pod="kube-system/global-pull-secret-syncer-kkcc6"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638294    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638318    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-cni-bin\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638330    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-system-cni-dir\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638357    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-system-cni-dir\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638392    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-cnibin\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:37.638420    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs podName:60bf362c-030f-4adf-9878-842b2fdc326b nodeName:}" failed. No retries permitted until 2026-10-30 05:31:38.138403155 +0000 UTC m=+1.883688465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs") pod "network-metrics-daemon-pfdj6" (UID: "60bf362c-030f-4adf-9878-842b2fdc326b") : secret "metrics-daemon-secret" not found
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638361    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e71a9464-fdd4-491d-a0ce-51519a35486d-cnibin\") pod \"multus-additional-cni-plugins-g6l5g\" (UID: \"e71a9464-fdd4-491d-a0ce-51519a35486d\") " pod="openshift-multus/multus-additional-cni-plugins-g6l5g"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638462    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-socket-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638486    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-sys-fs\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638515    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-system-cni-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638519    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-socket-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638538    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-run-ovn-kubernetes\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638569    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-host\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638578    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-sys-fs\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638612    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-registration-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638620    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-system-cni-dir\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638634    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638652    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4c2b487-ec63-4b07-a11c-781cb9afd37d-hosts-file\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638671    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638673    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-registration-dir\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638634    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-run-ovn-kubernetes\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638674    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/14a30f61-33a2-4f29-ba24-fc02c0c6cc15-host\") pod \"node-ca-qw6mh\" (UID: \"14a30f61-33a2-4f29-ba24-fc02c0c6cc15\") " pod="openshift-image-registry/node-ca-qw6mh"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638691    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-run-netns\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638731    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-host-run-netns\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638734    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b4c2b487-ec63-4b07-a11c-781cb9afd37d-hosts-file\") pod \"node-resolver-mtj5z\" (UID: \"b4c2b487-ec63-4b07-a11c-781cb9afd37d\") " pod="openshift-dns/node-resolver-mtj5z"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:37.638746    5066 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638751    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-node-log\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638777    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-kubernetes\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638781    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0421f010-8e05-4896-b171-301e85964bb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-24897\" (UID: \"0421f010-8e05-4896-b171-301e85964bb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24897"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:37.638805    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls podName:c46db00b-3c48-4d8d-844c-918d8126cc41 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:38.138790259 +0000 UTC m=+1.884075583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls") pod "dns-default-bzbfz" (UID: "c46db00b-3c48-4d8d-844c-918d8126cc41") : secret "dns-default-metrics-tls" not found
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638819    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-node-log\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638832    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-var-lib-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638860    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638867    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-kubernetes\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638890    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-netns\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638914    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-hostroot\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638939    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638895    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02a30d07-33cc-4be0-a920-8e839f8a493b-var-lib-openvswitch\") pod \"ovnkube-node-hpgtg\" (UID: \"02a30d07-33cc-4be0-a920-8e839f8a493b\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638965    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638990    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-run-netns\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.638993    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-cni-bin\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.639069    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-cni-multus\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.639108    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysctl-d\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.639139    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-kubelet\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:37.639199    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle podName:4684e0e6-736f-41bf-8f22-77245e339d58 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:38.139183061 +0000 UTC m=+1.884468405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle") pod "router-default-59857694c5-wq9fj" (UID: "4684e0e6-736f-41bf-8f22-77245e339d58") : configmap references non-existent config key: service-ca.crt
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.639217    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-kubelet\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.639196    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-cni-bin\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.639234    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-host-var-lib-cni-multus\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.639349    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d432e7e1-19d3-4f6d-8d47-51f8d6a3d166-etc-sysctl-d\") pod \"tuned-9gcnl\" (UID: \"d432e7e1-19d3-4f6d-8d47-51f8d6a3d166\") " pod="openshift-cluster-node-tuning-operator/tuned-9gcnl"
Oct 30 05:31:37.640421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.639351    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c2414f4-fa94-4054-9b41-b264e6088e67-hostroot\") pod \"multus-cbf62\" (UID: \"1c2414f4-fa94-4054-9b41-b264e6088e67\") " pod="openshift-multus/multus-cbf62"
Oct 30 05:31:37.642391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.641197    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/46c256fd-a89b-41f9-95aa-d8e81f783820-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qsjvb\" (UID: \"46c256fd-a89b-41f9-95aa-d8e81f783820\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:37.642391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.641195    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4684e0e6-736f-41bf-8f22-77245e339d58-metrics-certs\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:37.642391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.641272    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls\") pod \"image-registry-6b668646-vshtx\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") " pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:37.642391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.641927    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls\") pod \"image-registry-7d65554659-bdvbl\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") " pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.642391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.642079    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b0bdcd7b-f471-405a-87b2-ebedb8a1c950-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-v6b7k\" (UID: \"b0bdcd7b-f471-405a-87b2-ebedb8a1c950\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:37.687519 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.687490    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:37.692690 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.692658    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"
Oct 30 05:31:37.693327 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.693310    5066 scope.go:122] "RemoveContainer" containerID="b6927366bfdc148db909b2ffc636d98277afcb80bc1db53d386c12c71c939fb8"
Oct 30 05:31:37.693665 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.693644    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"
Oct 30 05:31:37.695803 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:37.695789    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:38.052700 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.052675    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d65554659-bdvbl"]
Oct 30 05:31:38.054959 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:38.054924    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode877e7c6_b2f6_4e79_ae22_2e04dcd3e76a.slice/crio-b71565eee140a2b6e4674e77d91af4fee9e283d4a05aff8e68a71d674ff4f983 WatchSource:0}: Error finding container b71565eee140a2b6e4674e77d91af4fee9e283d4a05aff8e68a71d674ff4f983: Status 404 returned error can't find the container with id b71565eee140a2b6e4674e77d91af4fee9e283d4a05aff8e68a71d674ff4f983
Oct 30 05:31:38.068004 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.067969    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb"]
Oct 30 05:31:38.120239 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.120220    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b668646-vshtx"]
Oct 30 05:31:38.121743 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:38.121717    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddac2ba5d_8990_47e3_aa65_02ff7500ac01.slice/crio-fe64e1e0a2e5840cde6a94c13e5c54dac4246fd5e259d0a09cf4543163237a13 WatchSource:0}: Error finding container fe64e1e0a2e5840cde6a94c13e5c54dac4246fd5e259d0a09cf4543163237a13: Status 404 returned error can't find the container with id fe64e1e0a2e5840cde6a94c13e5c54dac4246fd5e259d0a09cf4543163237a13
Oct 30 05:31:38.135078 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.135054    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-v6b7k"]
Oct 30 05:31:38.140458 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:38.140432    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0bdcd7b_f471_405a_87b2_ebedb8a1c950.slice/crio-d72cede87850656789036bf1fadd9bc756a3cbb6bdff1610001ea4d92a59cd1e WatchSource:0}: Error finding container d72cede87850656789036bf1fadd9bc756a3cbb6bdff1610001ea4d92a59cd1e: Status 404 returned error can't find the container with id d72cede87850656789036bf1fadd9bc756a3cbb6bdff1610001ea4d92a59cd1e
Oct 30 05:31:38.142760 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.142743    5066 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 05:31:38.144983 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.144955    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:38.145092 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.145007    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:38.145092 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.145047    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:38.145221 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.145096    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:38.145221 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.145145    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:38.146454 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:38.145498    5066 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 05:31:38.146454 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:38.145520    5066 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:38.146454 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:38.145558    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls podName:c46db00b-3c48-4d8d-844c-918d8126cc41 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:39.14553935 +0000 UTC m=+2.890824680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls") pod "dns-default-bzbfz" (UID: "c46db00b-3c48-4d8d-844c-918d8126cc41") : secret "dns-default-metrics-tls" not found
Oct 30 05:31:38.146454 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:38.145578    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls podName:ecfec034-dabf-45fb-a6cb-ef3ace810cb2 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:39.145567779 +0000 UTC m=+2.890853089 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6864g" (UID: "ecfec034-dabf-45fb-a6cb-ef3ace810cb2") : secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:38.146454 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.145833    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4684e0e6-736f-41bf-8f22-77245e339d58-service-ca-bundle\") pod \"router-default-59857694c5-wq9fj\" (UID: \"4684e0e6-736f-41bf-8f22-77245e339d58\") " pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:38.148998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.148976    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf362c-030f-4adf-9878-842b2fdc326b-metrics-certs\") pod \"network-metrics-daemon-pfdj6\" (UID: \"60bf362c-030f-4adf-9878-842b2fdc326b\") " pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:38.149145 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.149125    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bfa3e142-50e3-4b85-83fa-46f7dc8b7858-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6xnts\" (UID: \"bfa3e142-50e3-4b85-83fa-46f7dc8b7858\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:38.288972 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.288937    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:38.293027 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.293003    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"
Oct 30 05:31:38.295392 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.295340    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfdj6"
Oct 30 05:31:38.471625 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.471373    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b668646-vshtx" event={"ID":"dac2ba5d-8990-47e3-aa65-02ff7500ac01","Type":"ContainerStarted","Data":"bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a"}
Oct 30 05:31:38.471625 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.471444    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b668646-vshtx" event={"ID":"dac2ba5d-8990-47e3-aa65-02ff7500ac01","Type":"ContainerStarted","Data":"fe64e1e0a2e5840cde6a94c13e5c54dac4246fd5e259d0a09cf4543163237a13"}
Oct 30 05:31:38.473635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.473540    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d65554659-bdvbl" event={"ID":"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a","Type":"ContainerStarted","Data":"1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289"}
Oct 30 05:31:38.473635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.473621    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d65554659-bdvbl" event={"ID":"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a","Type":"ContainerStarted","Data":"b71565eee140a2b6e4674e77d91af4fee9e283d4a05aff8e68a71d674ff4f983"}
Oct 30 05:31:38.480152 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.480096    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k" event={"ID":"b0bdcd7b-f471-405a-87b2-ebedb8a1c950","Type":"ContainerStarted","Data":"d72cede87850656789036bf1fadd9bc756a3cbb6bdff1610001ea4d92a59cd1e"}
Oct 30 05:31:38.481500 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.481467    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb" event={"ID":"46c256fd-a89b-41f9-95aa-d8e81f783820","Type":"ContainerStarted","Data":"5bbe78c03d1ca3c646567d78aa9768b3499d2d42ebc30589d56abebe9600b787"}
Oct 30 05:31:38.484056 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.484034    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:31:38.484163 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.484144    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm" event={"ID":"ab824d2b-de9a-4384-a4a8-166f58808be8","Type":"ContainerStarted","Data":"82d88a24a32f6c0554bed0cce02680cf6621f885541e13874e1653874be9a3cb"}
Oct 30 05:31:38.578282 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.578248    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:38.632106 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.632069    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59857694c5-wq9fj"]
Oct 30 05:31:38.644546 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.644516    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts"]
Oct 30 05:31:38.656442 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:38.656417    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa3e142_50e3_4b85_83fa_46f7dc8b7858.slice/crio-2ef9f578a16cfbadcd9dba0034f24246d00a821426ae7c09cf72b87bd79efaed WatchSource:0}: Error finding container 2ef9f578a16cfbadcd9dba0034f24246d00a821426ae7c09cf72b87bd79efaed: Status 404 returned error can't find the container with id 2ef9f578a16cfbadcd9dba0034f24246d00a821426ae7c09cf72b87bd79efaed
Oct 30 05:31:38.667400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.667374    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pfdj6"]
Oct 30 05:31:38.685491 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.685441    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:31:38.755386 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.755333    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:38.915922 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:38.915841    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:39.075032 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:39.074946    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:39.155887 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:39.155854    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:39.156057 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:39.155903    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:39.156057 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:39.156020    5066 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:39.156178 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:39.156057    5066 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 05:31:39.156178 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:39.156090    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls podName:ecfec034-dabf-45fb-a6cb-ef3ace810cb2 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:41.156071214 +0000 UTC m=+4.901356543 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6864g" (UID: "ecfec034-dabf-45fb-a6cb-ef3ace810cb2") : secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:39.156178 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:39.156119    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls podName:c46db00b-3c48-4d8d-844c-918d8126cc41 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:41.156101771 +0000 UTC m=+4.901387096 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls") pod "dns-default-bzbfz" (UID: "c46db00b-3c48-4d8d-844c-918d8126cc41") : secret "dns-default-metrics-tls" not found
Oct 30 05:31:39.226491 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:39.226359    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:39.266790 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:39.265823    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:39.414562 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:39.414527    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:39.493124 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:39.493003    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pfdj6" event={"ID":"60bf362c-030f-4adf-9878-842b2fdc326b","Type":"ContainerStarted","Data":"b416d4ae81813353616c80a636f9f393816d58d8412096924ba978e8b996261a"}
Oct 30 05:31:39.497262 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:39.496994    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59857694c5-wq9fj" event={"ID":"4684e0e6-736f-41bf-8f22-77245e339d58","Type":"ContainerStarted","Data":"ae325b1f950469ba4bd83a35eab69f81a74c81b87d980bff6150671e7c246c8d"}
Oct 30 05:31:39.497262 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:39.497062    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59857694c5-wq9fj" event={"ID":"4684e0e6-736f-41bf-8f22-77245e339d58","Type":"ContainerStarted","Data":"dfe6a8186b24f37455e50fe21f6b391af0af57df1301dc6f2f1dd34fec57524f"}
Oct 30 05:31:39.500324 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:39.499764    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts" event={"ID":"bfa3e142-50e3-4b85-83fa-46f7dc8b7858","Type":"ContainerStarted","Data":"2ef9f578a16cfbadcd9dba0034f24246d00a821426ae7c09cf72b87bd79efaed"}
Oct 30 05:31:39.500324 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:39.499862    5066 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 05:31:40.289657 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.289627    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:40.292413 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.292391    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:40.512451 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.512413    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pfdj6" event={"ID":"60bf362c-030f-4adf-9878-842b2fdc326b","Type":"ContainerStarted","Data":"7ed4ce762069e9c6bb6a8731a821869b46e7a1c0bebf2909313ab6d057fa0340"}
Oct 30 05:31:40.512842 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.512525    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pfdj6" event={"ID":"60bf362c-030f-4adf-9878-842b2fdc326b","Type":"ContainerStarted","Data":"14c3de45b1fbe32865a8ef5720bc306bc69bfb84ac480cd2e0831bf54094fd63"}
Oct 30 05:31:40.514003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.513975    5066 generic.go:356] "Generic (PLEG): container finished" podID="0cf86020-761f-4fbc-808e-7dc6c5d5515f" containerID="7afadecf0987299fc16f626bb600d6a9ff4bf3f7355221acfd2a44fd16472be6" exitCode=0
Oct 30 05:31:40.514109 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.514056    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb" event={"ID":"0cf86020-761f-4fbc-808e-7dc6c5d5515f","Type":"ContainerDied","Data":"7afadecf0987299fc16f626bb600d6a9ff4bf3f7355221acfd2a44fd16472be6"}
Oct 30 05:31:40.514347 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.514329    5066 scope.go:122] "RemoveContainer" containerID="7afadecf0987299fc16f626bb600d6a9ff4bf3f7355221acfd2a44fd16472be6"
Oct 30 05:31:40.515525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.515498    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-v6b7k" event={"ID":"b0bdcd7b-f471-405a-87b2-ebedb8a1c950","Type":"ContainerStarted","Data":"06e358db53d661090e42186fa3032c7b5ef94b0ccd2430bce01af692cca32688"}
Oct 30 05:31:40.517421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.517396    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb" event={"ID":"46c256fd-a89b-41f9-95aa-d8e81f783820","Type":"ContainerStarted","Data":"2114b7eb55af1bce674d59acd054fbc492d076e1bd8d25bcabb69cfbe4f148fb"}
Oct 30 05:31:40.517516 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.517443    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qsjvb" event={"ID":"46c256fd-a89b-41f9-95aa-d8e81f783820","Type":"ContainerStarted","Data":"c3245c5aa637c7008b6c5fd96639cfbaff0ab4d27aa65092689b35924bd9795f"}
Oct 30 05:31:40.519000 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.518977    5066 generic.go:356] "Generic (PLEG): container finished" podID="86fadfbe-f000-4b42-a8ff-7b67061959c5" containerID="99e07dbd2d5e196e824a4da9856bb03b5e06e0dc1a5ca4ac3d9f6adef617b69d" exitCode=0
Oct 30 05:31:40.519084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.519051    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-swtbv" event={"ID":"86fadfbe-f000-4b42-a8ff-7b67061959c5","Type":"ContainerDied","Data":"99e07dbd2d5e196e824a4da9856bb03b5e06e0dc1a5ca4ac3d9f6adef617b69d"}
Oct 30 05:31:40.519377 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.519357    5066 scope.go:122] "RemoveContainer" containerID="99e07dbd2d5e196e824a4da9856bb03b5e06e0dc1a5ca4ac3d9f6adef617b69d"
Oct 30 05:31:40.520914 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.520894    5066 generic.go:356] "Generic (PLEG): container finished" podID="41edacc9-dea1-4343-96f5-5caf8a82fe84" containerID="c64589ee522efffc4d6b93cc1f4fe9eb739511511a4d51ae4a654bc60fa6a4d6" exitCode=0
Oct 30 05:31:40.520989 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.520974    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2" event={"ID":"41edacc9-dea1-4343-96f5-5caf8a82fe84","Type":"ContainerDied","Data":"c64589ee522efffc4d6b93cc1f4fe9eb739511511a4d51ae4a654bc60fa6a4d6"}
Oct 30 05:31:40.521415 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.521394    5066 scope.go:122] "RemoveContainer" containerID="c64589ee522efffc4d6b93cc1f4fe9eb739511511a4d51ae4a654bc60fa6a4d6"
Oct 30 05:31:40.583953 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.582510    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:40.786635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.786530    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:40.967231 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.967203    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:31:40.972987 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:40.972959    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:41.045633 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.045543    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:31:41.073232 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.073201    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:41.074885 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.074850    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-59857694c5-wq9fj"
Oct 30 05:31:41.153213 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.153169    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:41.160961 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.160936    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:41.161162 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.161150    5066 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 05:31:41.179390 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.179360    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:41.179573 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.179438    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:41.180002 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:41.179782    5066 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:41.180066 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:31:41.180055    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls podName:ecfec034-dabf-45fb-a6cb-ef3ace810cb2 nodeName:}" failed. No retries permitted until 2026-10-30 05:31:45.180038626 +0000 UTC m=+8.925323937 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6864g" (UID: "ecfec034-dabf-45fb-a6cb-ef3ace810cb2") : secret "insights-runtime-extractor-tls" not found
Oct 30 05:31:41.182844 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.182824    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c46db00b-3c48-4d8d-844c-918d8126cc41-metrics-tls\") pod \"dns-default-bzbfz\" (UID: \"c46db00b-3c48-4d8d-844c-918d8126cc41\") " pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:41.300678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.300583    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:41.308100 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.308075    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:41.467106 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.467071    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:41.527075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.527042    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-swtbv" event={"ID":"86fadfbe-f000-4b42-a8ff-7b67061959c5","Type":"ContainerStarted","Data":"dc133e01fe17dfe230f1fd4b554e30aa12473775011d8e59ade64628d6602bf6"}
Oct 30 05:31:41.529086 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.529063    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2fx2" event={"ID":"41edacc9-dea1-4343-96f5-5caf8a82fe84","Type":"ContainerStarted","Data":"29cbdf0b60795fef7123b20b1e95e9552f1c31f383a4d60d3782e326d15e9355"}
Oct 30 05:31:41.530834 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.530813    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-j55lb" event={"ID":"0cf86020-761f-4fbc-808e-7dc6c5d5515f","Type":"ContainerStarted","Data":"2b445828fd2e9a2742e0be3d90aa170eb11e47f17102b2486df0725cc70d1ca4"}
Oct 30 05:31:41.532391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.532365    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6xnts" event={"ID":"bfa3e142-50e3-4b85-83fa-46f7dc8b7858","Type":"ContainerStarted","Data":"5cacf8cc0686a8b452ad22a7abf8cdeb1e6d02c37e55270ed608f2aee44822bb"}
Oct 30 05:31:41.619757 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.619518    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bzbfz"]
Oct 30 05:31:41.622472 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:41.622443    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc46db00b_3c48_4d8d_844c_918d8126cc41.slice/crio-cb593de53a165431f7722c64e5d43dd0cf6848d4be097c614995085c4e8d0474 WatchSource:0}: Error finding container cb593de53a165431f7722c64e5d43dd0cf6848d4be097c614995085c4e8d0474: Status 404 returned error can't find the container with id cb593de53a165431f7722c64e5d43dd0cf6848d4be097c614995085c4e8d0474
Oct 30 05:31:41.635076 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.635055    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:41.944915 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:41.944879    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:42.102868 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:42.102831    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:42.277553 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:42.277474    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:42.277724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:42.277649    5066 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 05:31:42.283062 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:42.283036    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-58dpm"
Oct 30 05:31:42.538723 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:42.538637    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bzbfz" event={"ID":"c46db00b-3c48-4d8d-844c-918d8126cc41","Type":"ContainerStarted","Data":"cb593de53a165431f7722c64e5d43dd0cf6848d4be097c614995085c4e8d0474"}
Oct 30 05:31:42.576316 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:42.576283    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:42.732391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:42.732333    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:42.933242 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:42.933195    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:43.095525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:43.095489    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:43.251862 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:43.251836    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:43.407858 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:43.407819    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:43.544280 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:43.544193    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bzbfz" event={"ID":"c46db00b-3c48-4d8d-844c-918d8126cc41","Type":"ContainerStarted","Data":"40f491d1be24c1b8584e9c3c3ac6c1643cb2f4fff6a248dbb6caa1030c277847"}
Oct 30 05:31:43.544280 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:43.544252    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bzbfz" event={"ID":"c46db00b-3c48-4d8d-844c-918d8126cc41","Type":"ContainerStarted","Data":"c0fd485e10f902d0c0d781a1fe9e4ec1d3fd6fdfef5ff4e681f505904f845e60"}
Oct 30 05:31:43.544746 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:43.544360    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:44.177816 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:44.177777    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:44.375430 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:44.375400    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:44.544939 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:44.544861    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:44.727908 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:44.727875    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:45.217941 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:45.217897    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:45.220343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:45.220325    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecfec034-dabf-45fb-a6cb-ef3ace810cb2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6864g\" (UID: \"ecfec034-dabf-45fb-a6cb-ef3ace810cb2\") " pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:45.501666 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:45.501567    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6864g"
Oct 30 05:31:45.801414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:45.801390    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6864g"]
Oct 30 05:31:45.804175 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:45.804151    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecfec034_dabf_45fb_a6cb_ef3ace810cb2.slice/crio-e36c2ab6b6e739008749715a000a04be0169da53431428bff643e89facc24922 WatchSource:0}: Error finding container e36c2ab6b6e739008749715a000a04be0169da53431428bff643e89facc24922: Status 404 returned error can't find the container with id e36c2ab6b6e739008749715a000a04be0169da53431428bff643e89facc24922
Oct 30 05:31:46.558390 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:46.558347    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6864g" event={"ID":"ecfec034-dabf-45fb-a6cb-ef3ace810cb2","Type":"ContainerStarted","Data":"eda81e0adb83ab555b7b35f6574873d752886c4fe7bbf99c66aac39b22102518"}
Oct 30 05:31:46.558613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:46.558400    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6864g" event={"ID":"ecfec034-dabf-45fb-a6cb-ef3ace810cb2","Type":"ContainerStarted","Data":"0c024f62f69839d58c31080c93e39ee9732abded377b085c21fae63dda3dd383"}
Oct 30 05:31:46.558613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:46.558420    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6864g" event={"ID":"ecfec034-dabf-45fb-a6cb-ef3ace810cb2","Type":"ContainerStarted","Data":"e36c2ab6b6e739008749715a000a04be0169da53431428bff643e89facc24922"}
Oct 30 05:31:47.163652 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:47.163605    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:47.323503 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:47.323471    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:48.569280 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:48.569242    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6864g" event={"ID":"ecfec034-dabf-45fb-a6cb-ef3ace810cb2","Type":"ContainerStarted","Data":"b4e5e79c8bd5ca294ea9dfa535ce074edb60321a0cec9f453fe9546d27ce7d86"}
Oct 30 05:31:49.360876 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:49.360844    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpgtg"
Oct 30 05:31:53.057064 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.057023    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-gzmfz-hfdrp"]
Oct 30 05:31:53.059954 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.059934    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.072854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.072829    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7"]
Oct 30 05:31:53.075385 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.075362    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7"
Oct 30 05:31:53.086175 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.086147    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-archives-path\") pod \"periodic-gathering-gzmfz-hfdrp\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") " pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.086293 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.086193    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-service-ca-bundle\") pod \"periodic-gathering-gzmfz-hfdrp\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") " pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.086366 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.086287    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-trusted-ca-bundle\") pod \"periodic-gathering-gzmfz-hfdrp\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") " pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.086366 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.086344    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p6r7\" (UniqueName: \"kubernetes.io/projected/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-kube-api-access-8p6r7\") pod \"periodic-gathering-gzmfz-hfdrp\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") " pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.086463 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.086401    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6ab1819c-973d-4b13-a869-6a5f48af8a35-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-vnqv7\" (UID: \"6ab1819c-973d-4b13-a869-6a5f48af8a35\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7"
Oct 30 05:31:53.087388 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.087365    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-8xfrp"]
Oct 30 05:31:53.091716 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.091695    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-8xfrp"
Oct 30 05:31:53.092165 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.092140    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-bc768bb58-chngw"]
Oct 30 05:31:53.094296 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.094273    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.097385 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.097367    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 30 05:31:53.100971 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.100948    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-wkmvx\""
Oct 30 05:31:53.117268 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.117241    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d65554659-bdvbl"]
Oct 30 05:31:53.119420 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.118242    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7"]
Oct 30 05:31:53.122390 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.122358    5066 patch_prober.go:29] interesting pod/image-registry-7d65554659-bdvbl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 05:31:53.122533 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.122419    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7d65554659-bdvbl" podUID="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 05:31:53.125293 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.125270    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 05:31:53.125293 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.125275    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 05:31:53.126635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.126615    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 05:31:53.127031 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.127011    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-gzmfz-hfdrp"]
Oct 30 05:31:53.127994 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.127968    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-8xfrp"]
Oct 30 05:31:53.131492 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.131471    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 05:31:53.146950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.146927    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 05:31:53.160270 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.160235    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bc768bb58-chngw"]
Oct 30 05:31:53.186890 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.186861    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hrvz\" (UniqueName: \"kubernetes.io/projected/40e94164-3716-4461-8d31-f93b3a27c0e1-kube-api-access-4hrvz\") pod \"downloads-6dd789c8b7-8xfrp\" (UID: \"40e94164-3716-4461-8d31-f93b3a27c0e1\") " pod="openshift-console/downloads-6dd789c8b7-8xfrp"
Oct 30 05:31:53.187106 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.187080    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-archives-path\") pod \"periodic-gathering-gzmfz-hfdrp\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") " pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.187182 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.187132    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-service-ca-bundle\") pod \"periodic-gathering-gzmfz-hfdrp\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") " pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.187182 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.187156    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-trusted-ca-bundle\") pod \"periodic-gathering-gzmfz-hfdrp\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") " pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.187281 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.187186    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p6r7\" (UniqueName: \"kubernetes.io/projected/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-kube-api-access-8p6r7\") pod \"periodic-gathering-gzmfz-hfdrp\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") " pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.187281 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.187213    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6ab1819c-973d-4b13-a869-6a5f48af8a35-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-vnqv7\" (UID: \"6ab1819c-973d-4b13-a869-6a5f48af8a35\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7"
Oct 30 05:31:53.187281 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.187238    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40e94164-3716-4461-8d31-f93b3a27c0e1-tmp\") pod \"downloads-6dd789c8b7-8xfrp\" (UID: \"40e94164-3716-4461-8d31-f93b3a27c0e1\") " pod="openshift-console/downloads-6dd789c8b7-8xfrp"
Oct 30 05:31:53.187518 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.187482    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-archives-path\") pod \"periodic-gathering-gzmfz-hfdrp\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") " pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.187908 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.187887    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-service-ca-bundle\") pod \"periodic-gathering-gzmfz-hfdrp\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") " pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.188542 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.188515    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-trusted-ca-bundle\") pod \"periodic-gathering-gzmfz-hfdrp\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") " pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.190297 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.190274    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6ab1819c-973d-4b13-a869-6a5f48af8a35-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-vnqv7\" (UID: \"6ab1819c-973d-4b13-a869-6a5f48af8a35\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7"
Oct 30 05:31:53.194303 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.194278    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-7vthw\""
Oct 30 05:31:53.194303 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.194285    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 05:31:53.194472 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.194330    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fffdp\""
Oct 30 05:31:53.194472 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.194399    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 05:31:53.218850 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.218816    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:53.219018 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.219000    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6649cbddb-7n469"]
Oct 30 05:31:53.221739 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.221723    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.237689 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.237665    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p6r7\" (UniqueName: \"kubernetes.io/projected/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-kube-api-access-8p6r7\") pod \"periodic-gathering-gzmfz-hfdrp\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") " pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.251043 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.251015    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6649cbddb-7n469"]
Oct 30 05:31:53.288436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.288401    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-tmp\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.288436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.288443    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40e94164-3716-4461-8d31-f93b3a27c0e1-tmp\") pod \"downloads-6dd789c8b7-8xfrp\" (UID: \"40e94164-3716-4461-8d31-f93b3a27c0e1\") " pod="openshift-console/downloads-6dd789c8b7-8xfrp"
Oct 30 05:31:53.288730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.288463    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-registry-certificates\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.288730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.288666    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-config\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.288730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.288700    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg47d\" (UniqueName: \"kubernetes.io/projected/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-kube-api-access-xg47d\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.288896 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.288765    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v96b\" (UniqueName: \"kubernetes.io/projected/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-kube-api-access-4v96b\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.288896 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.288794    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-trusted-ca\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.288896 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.288813    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-service-ca\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.288896 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.288831    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-oauth-serving-cert\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.288896 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.288881    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40e94164-3716-4461-8d31-f93b3a27c0e1-tmp\") pod \"downloads-6dd789c8b7-8xfrp\" (UID: \"40e94164-3716-4461-8d31-f93b3a27c0e1\") " pod="openshift-console/downloads-6dd789c8b7-8xfrp"
Oct 30 05:31:53.289083 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.288900    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hrvz\" (UniqueName: \"kubernetes.io/projected/40e94164-3716-4461-8d31-f93b3a27c0e1-kube-api-access-4hrvz\") pod \"downloads-6dd789c8b7-8xfrp\" (UID: \"40e94164-3716-4461-8d31-f93b3a27c0e1\") " pod="openshift-console/downloads-6dd789c8b7-8xfrp"
Oct 30 05:31:53.289083 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.288928    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-bound-sa-token\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.289083 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.289021    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-image-registry-private-configuration\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.289083 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.289040    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-registry-tls\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.289222 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.289120    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-oauth-config\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.289222 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.289155    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-installation-pull-secrets\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.289222 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.289189    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-serving-cert\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.289347 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.289233    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-ca-trust-extracted\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.317930 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.317857    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hrvz\" (UniqueName: \"kubernetes.io/projected/40e94164-3716-4461-8d31-f93b3a27c0e1-kube-api-access-4hrvz\") pod \"downloads-6dd789c8b7-8xfrp\" (UID: \"40e94164-3716-4461-8d31-f93b3a27c0e1\") " pod="openshift-console/downloads-6dd789c8b7-8xfrp"
Oct 30 05:31:53.372147 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.372115    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:31:53.386995 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.386966    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7"
Oct 30 05:31:53.389627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389599    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-bound-sa-token\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.389761 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389655    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-image-registry-private-configuration\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.389761 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389678    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-registry-tls\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.389761 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389705    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-oauth-config\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.389761 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389722    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-installation-pull-secrets\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.389761 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389743    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-serving-cert\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.389761 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389764    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-ca-trust-extracted\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.390087 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389784    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-tmp\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.390087 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389816    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-registry-certificates\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.390087 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389843    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-config\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.390087 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389861    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg47d\" (UniqueName: \"kubernetes.io/projected/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-kube-api-access-xg47d\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.390087 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389879    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v96b\" (UniqueName: \"kubernetes.io/projected/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-kube-api-access-4v96b\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.390087 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389894    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-trusted-ca\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.390087 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389919    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-service-ca\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.390087 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.389942    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-oauth-serving-cert\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.390538 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.390445    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-tmp\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.390660 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.390638    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-oauth-serving-cert\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.390857 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.390827    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-ca-trust-extracted\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.391463 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.391357    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-registry-certificates\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.392633 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.391533    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-service-ca\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.392633 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.391794    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-config\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.392633 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.392506    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-trusted-ca\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.393007 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.392989    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-installation-pull-secrets\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.394526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.394500    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-registry-tls\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.394668 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.394553    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-image-registry-private-configuration\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.394992 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.394950    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-oauth-config\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.396532 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.396502    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-serving-cert\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.403576 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.403552    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-8xfrp"
Oct 30 05:31:53.415525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.415457    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-bound-sa-token\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.416628 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.416608    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v96b\" (UniqueName: \"kubernetes.io/projected/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-kube-api-access-4v96b\") pod \"console-bc768bb58-chngw\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") " pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.433814 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.433443    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg47d\" (UniqueName: \"kubernetes.io/projected/6e92e1cf-34ac-4b5b-a6c8-835ee435284c-kube-api-access-xg47d\") pod \"image-registry-6649cbddb-7n469\" (UID: \"6e92e1cf-34ac-4b5b-a6c8-835ee435284c\") " pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.440242 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.440087    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:53.532269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.532237    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:53.550314 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.550287    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bzbfz"
Oct 30 05:31:53.609344 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.609312    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:53.697438 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.697265    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-gzmfz-hfdrp"]
Oct 30 05:31:53.699986 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:53.699959    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ccf6bc_15a2_4779_bb91_3df7ed70d46a.slice/crio-d60b3a92890631ce6980089ae4f16c2db93979ba6feef71fe862b718a013be82 WatchSource:0}: Error finding container d60b3a92890631ce6980089ae4f16c2db93979ba6feef71fe862b718a013be82: Status 404 returned error can't find the container with id d60b3a92890631ce6980089ae4f16c2db93979ba6feef71fe862b718a013be82
Oct 30 05:31:53.708743 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.708714    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:31:53.727485 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.727465    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7"]
Oct 30 05:31:53.730779 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:53.730748    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ab1819c_973d_4b13_a869_6a5f48af8a35.slice/crio-3857529554bca469c151af53505c173a355d64919bb819601f3fb2715974107d WatchSource:0}: Error finding container 3857529554bca469c151af53505c173a355d64919bb819601f3fb2715974107d: Status 404 returned error can't find the container with id 3857529554bca469c151af53505c173a355d64919bb819601f3fb2715974107d
Oct 30 05:31:53.737972 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.737941    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-8xfrp"]
Oct 30 05:31:53.739251 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:53.739143    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e94164_3716_4461_8d31_f93b3a27c0e1.slice/crio-344beef326eb497c870efc95413158d778e78b8916b3f227b2fdb229b28b7075 WatchSource:0}: Error finding container 344beef326eb497c870efc95413158d778e78b8916b3f227b2fdb229b28b7075: Status 404 returned error can't find the container with id 344beef326eb497c870efc95413158d778e78b8916b3f227b2fdb229b28b7075
Oct 30 05:31:53.838238 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:53.838215    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6649cbddb-7n469"]
Oct 30 05:31:53.840208 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:53.840178    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e92e1cf_34ac_4b5b_a6c8_835ee435284c.slice/crio-687a82adfa3994098eef3a4f210c4f022932b547368b9ca99213ad339ee819a6 WatchSource:0}: Error finding container 687a82adfa3994098eef3a4f210c4f022932b547368b9ca99213ad339ee819a6: Status 404 returned error can't find the container with id 687a82adfa3994098eef3a4f210c4f022932b547368b9ca99213ad339ee819a6
Oct 30 05:31:54.019492 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.019457    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bc768bb58-chngw"]
Oct 30 05:31:54.022563 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:54.022535    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c5ec156_ed7f_4f31_8022_d5fd94e3144c.slice/crio-e0d829b433c41b0ca857452f9febef0be24a3e0fb03cbf18f7847e064d40d7e7 WatchSource:0}: Error finding container e0d829b433c41b0ca857452f9febef0be24a3e0fb03cbf18f7847e064d40d7e7: Status 404 returned error can't find the container with id e0d829b433c41b0ca857452f9febef0be24a3e0fb03cbf18f7847e064d40d7e7
Oct 30 05:31:54.600451 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.599638    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6649cbddb-7n469" event={"ID":"6e92e1cf-34ac-4b5b-a6c8-835ee435284c","Type":"ContainerStarted","Data":"cd491a51979f187f0ccbb86629f27053c53c5afd2a532042dcc762ada5d9d13f"}
Oct 30 05:31:54.600451 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.599703    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6649cbddb-7n469" event={"ID":"6e92e1cf-34ac-4b5b-a6c8-835ee435284c","Type":"ContainerStarted","Data":"687a82adfa3994098eef3a4f210c4f022932b547368b9ca99213ad339ee819a6"}
Oct 30 05:31:54.600451 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.600404    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:31:54.601929 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.601904    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-8xfrp" event={"ID":"40e94164-3716-4461-8d31-f93b3a27c0e1","Type":"ContainerStarted","Data":"344beef326eb497c870efc95413158d778e78b8916b3f227b2fdb229b28b7075"}
Oct 30 05:31:54.603668 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.603640    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7" event={"ID":"6ab1819c-973d-4b13-a869-6a5f48af8a35","Type":"ContainerStarted","Data":"3857529554bca469c151af53505c173a355d64919bb819601f3fb2715974107d"}
Oct 30 05:31:54.605732 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.605682    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc768bb58-chngw" event={"ID":"3c5ec156-ed7f-4f31-8022-d5fd94e3144c","Type":"ContainerStarted","Data":"e0d829b433c41b0ca857452f9febef0be24a3e0fb03cbf18f7847e064d40d7e7"}
Oct 30 05:31:54.609532 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.608045    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-gzmfz-hfdrp" event={"ID":"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a","Type":"ContainerStarted","Data":"06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3"}
Oct 30 05:31:54.609532 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.608085    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-gzmfz-hfdrp" event={"ID":"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a","Type":"ContainerStarted","Data":"d60b3a92890631ce6980089ae4f16c2db93979ba6feef71fe862b718a013be82"}
Oct 30 05:31:54.653492 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.651916    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6649cbddb-7n469" podStartSLOduration=1.6518988289999998 podStartE2EDuration="1.651898829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:54.651201919 +0000 UTC m=+18.396487253" watchObservedRunningTime="2026-10-30 05:31:54.651898829 +0000 UTC m=+18.397184161"
Oct 30 05:31:54.705450 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.705402    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-gzmfz-hfdrp" podStartSLOduration=1.705386849 podStartE2EDuration="1.705386849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:54.704953748 +0000 UTC m=+18.450239081" watchObservedRunningTime="2026-10-30 05:31:54.705386849 +0000 UTC m=+18.450672183"
Oct 30 05:31:54.719818 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.719780    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bzbfz_c46db00b-3c48-4d8d-844c-918d8126cc41/dns/0.log"
Oct 30 05:31:54.836981 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.836946    5066 patch_prober.go:29] interesting pod/image-registry-6b668646-vshtx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 05:31:54.837166 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.837005    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6b668646-vshtx" podUID="dac2ba5d-8990-47e3-aa65-02ff7500ac01" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 05:31:54.920521 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.920419    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bzbfz_c46db00b-3c48-4d8d-844c-918d8126cc41/kube-rbac-proxy/0.log"
Oct 30 05:31:54.967559 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:54.967446    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:55.320250 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:55.320171    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mtj5z_b4c2b487-ec63-4b07-a11c-781cb9afd37d/dns-node-resolver/0.log"
Oct 30 05:31:55.615922 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:55.615353    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7" event={"ID":"6ab1819c-973d-4b13-a869-6a5f48af8a35","Type":"ContainerStarted","Data":"0b3fee3857a0f79a6e6e9feac2c877ca6032213fa878730e7ff1bad08bff6c97"}
Oct 30 05:31:55.616289 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:55.616078    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7"
Oct 30 05:31:55.621693 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:55.621633    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7"
Oct 30 05:31:55.707275 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:55.707229    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-vnqv7" podStartSLOduration=1.5312792069999999 podStartE2EDuration="2.707214842s" totalImagesPullingTime="1.175935635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:55.661860997 +0000 UTC m=+19.407146330" watchObservedRunningTime="2026-10-30 05:31:55.707214842 +0000 UTC m=+19.452500177"
Oct 30 05:31:55.723375 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:55.723324    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6649cbddb-7n469_6e92e1cf-34ac-4b5b-a6c8-835ee435284c/registry/0.log"
Oct 30 05:31:55.920676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:55.920642    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6b668646-vshtx_dac2ba5d-8990-47e3-aa65-02ff7500ac01/registry/0.log"
Oct 30 05:31:56.121090 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.120997    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7d65554659-bdvbl_e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/registry/0.log"
Oct 30 05:31:56.503131 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.503098    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d5ddb484b-xlrs8"]
Oct 30 05:31:56.506119 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.506095    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.516184 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.516159    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-trusted-ca-bundle\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.516322 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.516229    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq6rd\" (UniqueName: \"kubernetes.io/projected/a7792bba-50cd-4e10-8894-3dbcb9204a70-kube-api-access-kq6rd\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.516382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.516342    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-oauth-config\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.516449 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.516429    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-service-ca\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.516501 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.516483    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-serving-cert\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.516621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.516561    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-config\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.516721 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.516627    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7792bba-50cd-4e10-8894-3dbcb9204a70-tmp\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.516721 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.516668    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-oauth-serving-cert\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.568952 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.568920    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 05:31:56.570568 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.570538    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d5ddb484b-xlrs8"]
Oct 30 05:31:56.617152 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.617105    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-trusted-ca-bundle\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.617766 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.617743    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq6rd\" (UniqueName: \"kubernetes.io/projected/a7792bba-50cd-4e10-8894-3dbcb9204a70-kube-api-access-kq6rd\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.617897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.617882    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-oauth-config\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.618027 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.618011    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-service-ca\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.618138 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.618124    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-serving-cert\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.618276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.618243    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-config\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.618276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.618275    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7792bba-50cd-4e10-8894-3dbcb9204a70-tmp\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.618476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.618320    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-oauth-serving-cert\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.619068 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.619044    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-oauth-serving-cert\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.619301 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.619279    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-config\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.619373 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.619344    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7792bba-50cd-4e10-8894-3dbcb9204a70-tmp\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.619496 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.619465    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-trusted-ca-bundle\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.619781 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.619761    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-service-ca\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.622249 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.622196    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-oauth-config\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.624478 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.624453    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-serving-cert\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.633189 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.633155    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:56.644522 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.644496    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq6rd\" (UniqueName: \"kubernetes.io/projected/a7792bba-50cd-4e10-8894-3dbcb9204a70-kube-api-access-kq6rd\") pod \"console-6d5ddb484b-xlrs8\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") " pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.699023 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.698992    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"]
Oct 30 05:31:56.705254 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.705226    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.719770 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.719739    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f8f7445-3608-45fe-8964-81fc8fb06e56-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-zzsn2\" (UID: \"6f8f7445-3608-45fe-8964-81fc8fb06e56\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.719998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.719820    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f8f7445-3608-45fe-8964-81fc8fb06e56-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-zzsn2\" (UID: \"6f8f7445-3608-45fe-8964-81fc8fb06e56\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.719998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.719840    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w9zj\" (UniqueName: \"kubernetes.io/projected/6f8f7445-3608-45fe-8964-81fc8fb06e56-kube-api-access-4w9zj\") pod \"prometheus-operator-9cd89b74c-zzsn2\" (UID: \"6f8f7445-3608-45fe-8964-81fc8fb06e56\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.719998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.719860    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f8f7445-3608-45fe-8964-81fc8fb06e56-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-zzsn2\" (UID: \"6f8f7445-3608-45fe-8964-81fc8fb06e56\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.724950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.723488    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"]
Oct 30 05:31:56.724950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.724176    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 05:31:56.724950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.724270    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 05:31:56.724950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.724385    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 05:31:56.724950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.724657    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qw6mh_14a30f61-33a2-4f29-ba24-fc02c0c6cc15/node-ca/0.log"
Oct 30 05:31:56.724950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.724882    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-crw9s\""
Oct 30 05:31:56.792083 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.792007    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:56.820959 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.820926    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:31:56.821148 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.821109    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f8f7445-3608-45fe-8964-81fc8fb06e56-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-zzsn2\" (UID: \"6f8f7445-3608-45fe-8964-81fc8fb06e56\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.821198 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.821173    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f8f7445-3608-45fe-8964-81fc8fb06e56-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-zzsn2\" (UID: \"6f8f7445-3608-45fe-8964-81fc8fb06e56\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.821243 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.821202    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w9zj\" (UniqueName: \"kubernetes.io/projected/6f8f7445-3608-45fe-8964-81fc8fb06e56-kube-api-access-4w9zj\") pod \"prometheus-operator-9cd89b74c-zzsn2\" (UID: \"6f8f7445-3608-45fe-8964-81fc8fb06e56\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.821275 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.821238    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f8f7445-3608-45fe-8964-81fc8fb06e56-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-zzsn2\" (UID: \"6f8f7445-3608-45fe-8964-81fc8fb06e56\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.822231 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.822202    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f8f7445-3608-45fe-8964-81fc8fb06e56-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-zzsn2\" (UID: \"6f8f7445-3608-45fe-8964-81fc8fb06e56\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.825272 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.825225    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f8f7445-3608-45fe-8964-81fc8fb06e56-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-zzsn2\" (UID: \"6f8f7445-3608-45fe-8964-81fc8fb06e56\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.826194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.826172    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f8f7445-3608-45fe-8964-81fc8fb06e56-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-zzsn2\" (UID: \"6f8f7445-3608-45fe-8964-81fc8fb06e56\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.848752 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.848723    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w9zj\" (UniqueName: \"kubernetes.io/projected/6f8f7445-3608-45fe-8964-81fc8fb06e56-kube-api-access-4w9zj\") pod \"prometheus-operator-9cd89b74c-zzsn2\" (UID: \"6f8f7445-3608-45fe-8964-81fc8fb06e56\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:56.920560 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.920533    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59857694c5-wq9fj_4684e0e6-736f-41bf-8f22-77245e339d58/router/0.log"
Oct 30 05:31:56.949307 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:56.949270    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:57.036685 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:57.036635    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"
Oct 30 05:31:57.119861 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:57.119831    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mfzzm_a655cfec-4ee3-4ac9-badc-4fb318603506/serve-healthcheck-canary/0.log"
Oct 30 05:31:57.136887 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:57.136850    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d5ddb484b-xlrs8"]
Oct 30 05:31:57.354767 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:57.354688    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2"]
Oct 30 05:31:57.359198 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:31:57.359155    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f8f7445_3608_45fe_8964_81fc8fb06e56.slice/crio-0881563c47e9fd2339210d0192ab128021c474c1c5b506a95790d61b23e79166 WatchSource:0}: Error finding container 0881563c47e9fd2339210d0192ab128021c474c1c5b506a95790d61b23e79166: Status 404 returned error can't find the container with id 0881563c47e9fd2339210d0192ab128021c474c1c5b506a95790d61b23e79166
Oct 30 05:31:57.511204 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:57.511176    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:57.629746 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:57.629665    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d5ddb484b-xlrs8" event={"ID":"a7792bba-50cd-4e10-8894-3dbcb9204a70","Type":"ContainerStarted","Data":"64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842"}
Oct 30 05:31:57.629746 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:57.629730    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d5ddb484b-xlrs8" event={"ID":"a7792bba-50cd-4e10-8894-3dbcb9204a70","Type":"ContainerStarted","Data":"503f81586a69525e8c30f08f9bd47af5a44f2ce3a7b9f8d755ab77dad5b98a46"}
Oct 30 05:31:57.631876 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:57.631845    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc768bb58-chngw" event={"ID":"3c5ec156-ed7f-4f31-8022-d5fd94e3144c","Type":"ContainerStarted","Data":"c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71"}
Oct 30 05:31:57.633236 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:57.633212    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2" event={"ID":"6f8f7445-3608-45fe-8964-81fc8fb06e56","Type":"ContainerStarted","Data":"0881563c47e9fd2339210d0192ab128021c474c1c5b506a95790d61b23e79166"}
Oct 30 05:31:57.670404 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:57.670371    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:57.679998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:57.679936    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6d5ddb484b-xlrs8" podStartSLOduration=1.352560647 podStartE2EDuration="1.679915946s" totalImagesPullingTime="327.355299ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:31:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:57.677717526 +0000 UTC m=+21.423002858" watchObservedRunningTime="2026-10-30 05:31:57.679915946 +0000 UTC m=+21.425201282"
Oct 30 05:31:57.725494 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:57.725436    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-bc768bb58-chngw" podStartSLOduration=1.562935236 podStartE2EDuration="4.725420935s" totalImagesPullingTime="3.162485699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:57.724499917 +0000 UTC m=+21.469785261" watchObservedRunningTime="2026-10-30 05:31:57.725420935 +0000 UTC m=+21.470706266"
Oct 30 05:31:58.650905 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:58.650862    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2" event={"ID":"6f8f7445-3608-45fe-8964-81fc8fb06e56","Type":"ContainerStarted","Data":"26d0bd40dcd280220578eca7764f164a9f13730481acf25653c29fbd4ac3395d"}
Oct 30 05:31:59.555353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:59.555245    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:31:59.659883 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:59.659842    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2" event={"ID":"6f8f7445-3608-45fe-8964-81fc8fb06e56","Type":"ContainerStarted","Data":"7934d1c38e498490224f041dc807003d4966af7affbef97ee7bc8c3fd3d069fa"}
Oct 30 05:31:59.707250 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:31:59.707183    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-zzsn2" podStartSLOduration=2.528926023 podStartE2EDuration="3.707161152s" totalImagesPullingTime="1.178235129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:31:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:31:59.706361501 +0000 UTC m=+23.451646835" watchObservedRunningTime="2026-10-30 05:31:59.707161152 +0000 UTC m=+23.452446482"
Oct 30 05:32:00.124065 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:00.124035    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bfpdf"
Oct 30 05:32:00.124870 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:00.124843    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:00.280857 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:00.280818    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:00.527119 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:00.527088    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:32:00.913951 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:00.913907    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:01.072436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:01.072385    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:02.150401 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.150371    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:02.343240 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.342531    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zdfdw"]
Oct 30 05:32:02.354262 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.354228    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:02.387909 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.387874    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.439866 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.439833    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 05:32:02.440069 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.440015    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 05:32:02.440154 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.440101    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lg7qg\""
Oct 30 05:32:02.440154 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.440146    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 05:32:02.476096 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.476066    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.476285 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.476109    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-metrics-client-ca\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.476285 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.476148    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-tls\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.476285 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.476233    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-sys\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.476285 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.476279    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-wtmp\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.476457 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.476308    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-accelerators-collector-config\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.476457 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.476374    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b45jq\" (UniqueName: \"kubernetes.io/projected/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-kube-api-access-b45jq\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.476457 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.476412    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-textfile\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.476667 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.476457    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-root\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.519541 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.519489    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:02.556647 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:32:02.556584    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:32:02.577262 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.577231    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-root\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.577461 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.577311    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.577461 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.577344    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-metrics-client-ca\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.577461 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.577364    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-root\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.577461 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.577384    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-tls\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.577461 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.577429    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-sys\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.577461 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.577456    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-wtmp\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.577779 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.577481    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-accelerators-collector-config\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.577779 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.577514    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b45jq\" (UniqueName: \"kubernetes.io/projected/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-kube-api-access-b45jq\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.577779 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.577561    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-textfile\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.577779 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.577565    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-sys\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.577779 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.577676    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-wtmp\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.578155 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.578132    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-textfile\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.578364 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.578327    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-accelerators-collector-config\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.578364 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.578357    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-metrics-client-ca\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.581201 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.581173    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.581301 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.581251    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-node-exporter-tls\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.603678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.603648    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b45jq\" (UniqueName: \"kubernetes.io/projected/5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4-kube-api-access-b45jq\") pod \"node-exporter-zdfdw\" (UID: \"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4\") " pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.688279 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.688245    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:02.764321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.764248    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zdfdw"
Oct 30 05:32:02.800936 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:32:02.800902    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5447ce87_ff6b_4a21_bb0b_3bf21d26b0a4.slice/crio-7c75deaeba2253cde29d01ba8acd72760fc7f21105d9b2a318b8cfcc840e794b WatchSource:0}: Error finding container 7c75deaeba2253cde29d01ba8acd72760fc7f21105d9b2a318b8cfcc840e794b: Status 404 returned error can't find the container with id 7c75deaeba2253cde29d01ba8acd72760fc7f21105d9b2a318b8cfcc840e794b
Oct 30 05:32:02.847997 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:02.847958    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:03.031500 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:03.031419    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:03.123790 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:03.123757    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:32:03.557846 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:03.557798    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:03.685677 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:03.685638    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zdfdw" event={"ID":"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4","Type":"ContainerStarted","Data":"7c75deaeba2253cde29d01ba8acd72760fc7f21105d9b2a318b8cfcc840e794b"}
Oct 30 05:32:03.709112 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:03.709076    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:32:03.710108 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:03.710029    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:32:03.716635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:03.716607    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:32:03.720368 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:03.720338    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:03.891895 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:03.891823    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:04.693425 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:04.693387    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zdfdw" event={"ID":"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4","Type":"ContainerStarted","Data":"7fa250461accd063dc88fb34d2d6daa3b351d8e36fd60270e74a8e84c15fbb77"}
Oct 30 05:32:04.698273 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:04.698246    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:32:05.613964 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:05.613922    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:05.702625 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:05.702494    5066 generic.go:356] "Generic (PLEG): container finished" podID="5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4" containerID="7fa250461accd063dc88fb34d2d6daa3b351d8e36fd60270e74a8e84c15fbb77" exitCode=0
Oct 30 05:32:05.702625 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:05.702538    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zdfdw" event={"ID":"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4","Type":"ContainerDied","Data":"7fa250461accd063dc88fb34d2d6daa3b351d8e36fd60270e74a8e84c15fbb77"}
Oct 30 05:32:05.772552 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:05.772518    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:06.184182 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:06.184138    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:06.711397 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:06.711360    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zdfdw" event={"ID":"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4","Type":"ContainerStarted","Data":"61f0eda31f54d9dad87bc39b243cca18379a843ae6c294aed323bb271cd35e06"}
Oct 30 05:32:06.711899 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:06.711428    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zdfdw" event={"ID":"5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4","Type":"ContainerStarted","Data":"7b8e9499b4cd871d189e837da5d954c2f899f5d562368294eb512cd1bd12f621"}
Oct 30 05:32:06.789367 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:06.789308    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zdfdw" podStartSLOduration=2.9835589049999998 podStartE2EDuration="4.789289261s" totalImagesPullingTime="1.698081763s" totalInitContainerRuntime="107.648593ms" isStatefulPod=true podCreationTimestamp="2026-10-30 05:32:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:32:06.788639477 +0000 UTC m=+30.533924809" watchObservedRunningTime="2026-10-30 05:32:06.789289261 +0000 UTC m=+30.534574597"
Oct 30 05:32:06.822239 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:06.822206    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:32:06.822400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:06.822272    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:32:06.827693 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:06.827668    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:32:07.153402 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:07.153373    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:07.370941 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:07.370894    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:07.551679 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:07.551556    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:07.721214 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:07.721181    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:32:07.800869 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:07.800831    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bc768bb58-chngw"]
Oct 30 05:32:09.201264 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:09.201190    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:09.364473 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:09.364384    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:09.519814 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:09.519716    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:10.970172 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:10.970144    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fdfg7"
Oct 30 05:32:11.242580 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:11.242502    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:11.402692 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:11.402666    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:11.739673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:11.739624    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-8xfrp" event={"ID":"40e94164-3716-4461-8d31-f93b3a27c0e1","Type":"ContainerStarted","Data":"1c0601242952f41ba2e64227c1833342f84b795e096e1d80807fb78a2f137c73"}
Oct 30 05:32:11.739864 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:11.739820    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-8xfrp"
Oct 30 05:32:11.741563 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:11.741541    5066 patch_prober.go:29] interesting pod/downloads-6dd789c8b7-8xfrp container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.30:8080/\": dial tcp 10.132.0.30:8080: connect: connection refused" start-of-body=
Oct 30 05:32:11.741673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:11.741604    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6dd789c8b7-8xfrp" podUID="40e94164-3716-4461-8d31-f93b3a27c0e1" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.30:8080/\": dial tcp 10.132.0.30:8080: connect: connection refused"
Oct 30 05:32:11.806208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:11.806161    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-8xfrp" podStartSLOduration=1.163284795 podStartE2EDuration="18.806147673s" totalImagesPullingTime="17.642862878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:32:11.800203968 +0000 UTC m=+35.545489300" watchObservedRunningTime="2026-10-30 05:32:11.806147673 +0000 UTC m=+35.551433005"
Oct 30 05:32:12.758060 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:12.758030    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-8xfrp"
Oct 30 05:32:13.729224 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.729190    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b4cccf476-wkhc5"]
Oct 30 05:32:13.760373 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.760307    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.772858 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.764876    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b4cccf476-wkhc5"]
Oct 30 05:32:13.790261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.789790    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-trusted-ca-bundle\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.790261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.789834    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-oauth-config\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.790261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.789870    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a10f6321-4f07-4e27-bf4a-5113e0746b01-tmp\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.790261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.789901    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-service-ca\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.790261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.789931    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtrfh\" (UniqueName: \"kubernetes.io/projected/a10f6321-4f07-4e27-bf4a-5113e0746b01-kube-api-access-rtrfh\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.790261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.789961    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-oauth-serving-cert\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.790261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.790047    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-serving-cert\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.790261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.790071    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-config\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.891102 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.891061    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-trusted-ca-bundle\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.891276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.891113    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-oauth-config\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.891276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.891136    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a10f6321-4f07-4e27-bf4a-5113e0746b01-tmp\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.891276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.891161    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-service-ca\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.891276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.891193    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtrfh\" (UniqueName: \"kubernetes.io/projected/a10f6321-4f07-4e27-bf4a-5113e0746b01-kube-api-access-rtrfh\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.891276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.891224    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-oauth-serving-cert\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.891508 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.891288    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-serving-cert\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.891508 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.891306    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-config\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.892054 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.892035    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-config\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.893123 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.893098    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-trusted-ca-bundle\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.893293 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.893216    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-service-ca\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.893541 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.893521    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a10f6321-4f07-4e27-bf4a-5113e0746b01-tmp\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.893796 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.893775    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-oauth-serving-cert\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.896093 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.896072    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-serving-cert\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.896827 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.896804    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-oauth-config\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:13.919651 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:13.919581    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtrfh\" (UniqueName: \"kubernetes.io/projected/a10f6321-4f07-4e27-bf4a-5113e0746b01-kube-api-access-rtrfh\") pod \"console-5b4cccf476-wkhc5\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") " pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:14.112157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:14.112069    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:14.366466 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:14.366364    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:14.479705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:14.479672    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b4cccf476-wkhc5"]
Oct 30 05:32:14.593778 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:14.590722    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:14.796328 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:14.795580    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4cccf476-wkhc5" event={"ID":"a10f6321-4f07-4e27-bf4a-5113e0746b01","Type":"ContainerStarted","Data":"4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0"}
Oct 30 05:32:14.796328 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:14.795693    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4cccf476-wkhc5" event={"ID":"a10f6321-4f07-4e27-bf4a-5113e0746b01","Type":"ContainerStarted","Data":"a5b6aa547d642b1fd43db0bf12117ff23f69826d2cd17e2ed2c5e2efdcb0773c"}
Oct 30 05:32:14.851013 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:14.850941    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5b4cccf476-wkhc5" podStartSLOduration=1.850920857 podStartE2EDuration="1.850920857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:32:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:32:14.850008235 +0000 UTC m=+38.595293570" watchObservedRunningTime="2026-10-30 05:32:14.850920857 +0000 UTC m=+38.596206207"
Oct 30 05:32:16.626673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:16.626572    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6649cbddb-7n469"
Oct 30 05:32:16.729638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:16.729575    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b668646-vshtx"]
Oct 30 05:32:16.894542 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:16.894467    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:17.054519 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:17.054385    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:17.215332 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:17.215303    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:17.407002 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:17.406964    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:18.142545 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.142497    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d65554659-bdvbl" podUID="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" containerName="registry" containerID="cri-o://1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289" gracePeriod=30
Oct 30 05:32:18.451397 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.451366    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:32:18.571496 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.571459    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-ca-trust-extracted\") pod \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") "
Oct 30 05:32:18.571724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.571535    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-image-registry-private-configuration\") pod \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") "
Oct 30 05:32:18.571724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.571637    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-kube-api-access-bptqj\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-kube-api-access-bptqj\") pod \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") "
Oct 30 05:32:18.571724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.571686    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-installation-pull-secrets\") pod \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") "
Oct 30 05:32:18.571724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.571714    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls\") pod \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") "
Oct 30 05:32:18.571932 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.571745    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-bound-sa-token\") pod \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") "
Oct 30 05:32:18.571932 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.571774    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-certificates\") pod \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") "
Oct 30 05:32:18.571932 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.571851    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-trusted-ca\") pod \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\" (UID: \"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a\") "
Oct 30 05:32:18.572531 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.572505    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-trusted-ca" pod "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:18.572971 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.572909    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-certificates" pod "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:18.574963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.574929    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-kube-api-access-bptqj" pod "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a"). InnerVolumeSpecName "kube-api-access-bptqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:32:18.581617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.576660    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-image-registry-private-configuration" pod "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:32:18.581617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.576884    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-installation-pull-secrets" pod "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:32:18.581617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.576952    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls" pod "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:32:18.581617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.577140    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-bound-sa-token" pod "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:32:18.581617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.580580    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-ca-trust-extracted" pod "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" (UID: "e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:32:18.672822 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.672748    5066 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-ca-trust-extracted\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:18.672822 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.672779    5066 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-image-registry-private-configuration\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:18.672822 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.672791    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bptqj\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-kube-api-access-bptqj\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:18.672822 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.672800    5066 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-installation-pull-secrets\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:18.672822 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.672809    5066 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:18.672822 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.672823    5066 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-bound-sa-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:18.673167 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.672831    5066 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-registry-certificates\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:18.673167 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.672839    5066 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a-trusted-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:18.830984 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.830947    5066 generic.go:356] "Generic (PLEG): container finished" podID="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" containerID="1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289" exitCode=0
Oct 30 05:32:18.831152 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.831010    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d65554659-bdvbl"
Oct 30 05:32:18.831152 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.831018    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d65554659-bdvbl" event={"ID":"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a","Type":"ContainerDied","Data":"1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289"}
Oct 30 05:32:18.831152 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.831074    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d65554659-bdvbl" event={"ID":"e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a","Type":"ContainerDied","Data":"b71565eee140a2b6e4674e77d91af4fee9e283d4a05aff8e68a71d674ff4f983"}
Oct 30 05:32:18.831152 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.831100    5066 scope.go:122] "RemoveContainer" containerID="1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289"
Oct 30 05:32:18.842815 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.842793    5066 scope.go:122] "RemoveContainer" containerID="1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289"
Oct 30 05:32:18.843143 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:32:18.843120    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289\": container with ID starting with 1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289 not found: ID does not exist" containerID="1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289"
Oct 30 05:32:18.843240 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.843151    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289"} err="failed to get container status \"1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289\": rpc error: code = NotFound desc = could not find container \"1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289\": container with ID starting with 1ad2c87c5d50035f0d938ffeaaec352d4f06e26490904457d5bffa3155662289 not found: ID does not exist"
Oct 30 05:32:18.890469 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.890440    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d65554659-bdvbl"]
Oct 30 05:32:18.905126 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:18.905095    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d65554659-bdvbl"]
Oct 30 05:32:20.400120 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:20.400088    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" path="/var/lib/kubelet/pods/e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a/volumes"
Oct 30 05:32:22.015464 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:22.015428    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:22.201544 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:22.201510    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:22.365641 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:22.365531    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:24.112307 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:24.112267    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:24.112876 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:24.112340    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:24.117216 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:24.117195    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:24.860660 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:24.860631    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:32:24.943085 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:24.943052    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d5ddb484b-xlrs8"]
Oct 30 05:32:25.477115 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:25.477082    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:25.683671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:25.683638    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:25.694374 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:25.694350    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-4pjwt-q72n8"]
Oct 30 05:32:25.694544 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:25.694526    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-4pjwt-q72n8" podUID="f74041a6-7409-439c-9ecb-4bc8dcc2006e" containerName="insights-gathering" containerID="cri-o://ac3eacc39e302d2292f01fd39af46ebdff774d7f1cec718e23e931c257f108da" gracePeriod=30
Oct 30 05:32:25.862858 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:25.862770    5066 generic.go:356] "Generic (PLEG): container finished" podID="f74041a6-7409-439c-9ecb-4bc8dcc2006e" containerID="ac3eacc39e302d2292f01fd39af46ebdff774d7f1cec718e23e931c257f108da" exitCode=2
Oct 30 05:32:25.863029 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:25.862857    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-4pjwt-q72n8" event={"ID":"f74041a6-7409-439c-9ecb-4bc8dcc2006e","Type":"ContainerDied","Data":"ac3eacc39e302d2292f01fd39af46ebdff774d7f1cec718e23e931c257f108da"}
Oct 30 05:32:26.024071 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.024043    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:32:26.139936 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.139854    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f74041a6-7409-439c-9ecb-4bc8dcc2006e-kube-api-access-rfxnt\" (UniqueName: \"kubernetes.io/projected/f74041a6-7409-439c-9ecb-4bc8dcc2006e-kube-api-access-rfxnt\") pod \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") "
Oct 30 05:32:26.139936 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.139907    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-trusted-ca-bundle\") pod \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") "
Oct 30 05:32:26.139936 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.139926    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-service-ca-bundle\") pod \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") "
Oct 30 05:32:26.140166 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.139969    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f74041a6-7409-439c-9ecb-4bc8dcc2006e-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/f74041a6-7409-439c-9ecb-4bc8dcc2006e-archives-path\") pod \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\" (UID: \"f74041a6-7409-439c-9ecb-4bc8dcc2006e\") "
Oct 30 05:32:26.140355 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.140327    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-trusted-ca-bundle" pod "f74041a6-7409-439c-9ecb-4bc8dcc2006e" (UID: "f74041a6-7409-439c-9ecb-4bc8dcc2006e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:26.140355 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.140338    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-service-ca-bundle" pod "f74041a6-7409-439c-9ecb-4bc8dcc2006e" (UID: "f74041a6-7409-439c-9ecb-4bc8dcc2006e"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:26.140355 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.140340    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f74041a6-7409-439c-9ecb-4bc8dcc2006e-archives-path" pod "f74041a6-7409-439c-9ecb-4bc8dcc2006e" (UID: "f74041a6-7409-439c-9ecb-4bc8dcc2006e"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:32:26.142450 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.142428    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f74041a6-7409-439c-9ecb-4bc8dcc2006e-kube-api-access-rfxnt" pod "f74041a6-7409-439c-9ecb-4bc8dcc2006e" (UID: "f74041a6-7409-439c-9ecb-4bc8dcc2006e"). InnerVolumeSpecName "kube-api-access-rfxnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:32:26.241070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.241039    5066 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-trusted-ca-bundle\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:26.241070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.241065    5066 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f74041a6-7409-439c-9ecb-4bc8dcc2006e-service-ca-bundle\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:26.241070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.241074    5066 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/f74041a6-7409-439c-9ecb-4bc8dcc2006e-archives-path\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:26.241301 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.241083    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfxnt\" (UniqueName: \"kubernetes.io/projected/f74041a6-7409-439c-9ecb-4bc8dcc2006e-kube-api-access-rfxnt\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:26.868856 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.868778    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-4pjwt-q72n8"
Oct 30 05:32:26.868856 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.868782    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-4pjwt-q72n8" event={"ID":"f74041a6-7409-439c-9ecb-4bc8dcc2006e","Type":"ContainerDied","Data":"a5112a1b37f0d6d31f1c8c74496d6cc647a08bf2457a218f0b932b903ddd4728"}
Oct 30 05:32:26.868856 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.868838    5066 scope.go:122] "RemoveContainer" containerID="ac3eacc39e302d2292f01fd39af46ebdff774d7f1cec718e23e931c257f108da"
Oct 30 05:32:26.926748 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.926715    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-4pjwt-q72n8"]
Oct 30 05:32:26.941249 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:26.941221    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-4pjwt-q72n8"]
Oct 30 05:32:28.399778 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:28.399747    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f74041a6-7409-439c-9ecb-4bc8dcc2006e" path="/var/lib/kubelet/pods/f74041a6-7409-439c-9ecb-4bc8dcc2006e/volumes"
Oct 30 05:32:32.155211 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:32.155153    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:32.331461 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:32.331429    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:32.551546 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:32:32.551462    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:32:32.826934 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:32.826849    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-bc768bb58-chngw" podUID="3c5ec156-ed7f-4f31-8022-d5fd94e3144c" containerName="console" containerID="cri-o://c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71" gracePeriod=15
Oct 30 05:32:33.112012 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.111985    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:32:33.205942 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.205912    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-service-ca\") pod \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") "
Oct 30 05:32:33.206399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.205975    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-oauth-config\") pod \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") "
Oct 30 05:32:33.206399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.206018    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-tmp\") pod \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") "
Oct 30 05:32:33.206399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.206143    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-config\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-config\") pod \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") "
Oct 30 05:32:33.206399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.206197    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-kube-api-access-4v96b\" (UniqueName: \"kubernetes.io/projected/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-kube-api-access-4v96b\") pod \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") "
Oct 30 05:32:33.206399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.206223    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-serving-cert\") pod \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") "
Oct 30 05:32:33.206399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.206248    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-oauth-serving-cert\") pod \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\" (UID: \"3c5ec156-ed7f-4f31-8022-d5fd94e3144c\") "
Oct 30 05:32:33.206399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.206310    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-tmp" pod "3c5ec156-ed7f-4f31-8022-d5fd94e3144c" (UID: "3c5ec156-ed7f-4f31-8022-d5fd94e3144c"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:32:33.206958 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.206414    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-service-ca" pod "3c5ec156-ed7f-4f31-8022-d5fd94e3144c" (UID: "3c5ec156-ed7f-4f31-8022-d5fd94e3144c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:33.206958 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.206494    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-config" pod "3c5ec156-ed7f-4f31-8022-d5fd94e3144c" (UID: "3c5ec156-ed7f-4f31-8022-d5fd94e3144c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:33.206958 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.206572    5066 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:33.206958 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.206626    5066 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-tmp\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:33.206958 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.206645    5066 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-config\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:33.206958 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.206715    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-oauth-serving-cert" pod "3c5ec156-ed7f-4f31-8022-d5fd94e3144c" (UID: "3c5ec156-ed7f-4f31-8022-d5fd94e3144c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:33.208577 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.208538    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-oauth-config" pod "3c5ec156-ed7f-4f31-8022-d5fd94e3144c" (UID: "3c5ec156-ed7f-4f31-8022-d5fd94e3144c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:32:33.208577 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.208554    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-kube-api-access-4v96b" pod "3c5ec156-ed7f-4f31-8022-d5fd94e3144c" (UID: "3c5ec156-ed7f-4f31-8022-d5fd94e3144c"). InnerVolumeSpecName "kube-api-access-4v96b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:32:33.208732 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.208560    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-serving-cert" pod "3c5ec156-ed7f-4f31-8022-d5fd94e3144c" (UID: "3c5ec156-ed7f-4f31-8022-d5fd94e3144c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:32:33.307360 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.307326    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v96b\" (UniqueName: \"kubernetes.io/projected/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-kube-api-access-4v96b\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:33.307360 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.307353    5066 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-serving-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:33.307360 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.307364    5066 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-oauth-serving-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:33.307360 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.307374    5066 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c5ec156-ed7f-4f31-8022-d5fd94e3144c-console-oauth-config\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:33.905631 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.905570    5066 generic.go:356] "Generic (PLEG): container finished" podID="3c5ec156-ed7f-4f31-8022-d5fd94e3144c" containerID="c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71" exitCode=0
Oct 30 05:32:33.905836 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.905648    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc768bb58-chngw"
Oct 30 05:32:33.905836 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.905664    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc768bb58-chngw" event={"ID":"3c5ec156-ed7f-4f31-8022-d5fd94e3144c","Type":"ContainerDied","Data":"c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71"}
Oct 30 05:32:33.905836 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.905734    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc768bb58-chngw" event={"ID":"3c5ec156-ed7f-4f31-8022-d5fd94e3144c","Type":"ContainerDied","Data":"e0d829b433c41b0ca857452f9febef0be24a3e0fb03cbf18f7847e064d40d7e7"}
Oct 30 05:32:33.905836 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.905761    5066 scope.go:122] "RemoveContainer" containerID="c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71"
Oct 30 05:32:33.914880 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.914863    5066 scope.go:122] "RemoveContainer" containerID="c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71"
Oct 30 05:32:33.915178 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:32:33.915159    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71\": container with ID starting with c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71 not found: ID does not exist" containerID="c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71"
Oct 30 05:32:33.915228 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.915189    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71"} err="failed to get container status \"c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71\": rpc error: code = NotFound desc = could not find container \"c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71\": container with ID starting with c4e8af5d78f7f0973bb5e1fb88f7abee0e4cc0d6ba9cf38673c221c5925c4d71 not found: ID does not exist"
Oct 30 05:32:33.963921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.963890    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bc768bb58-chngw"]
Oct 30 05:32:33.978350 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:33.978325    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bc768bb58-chngw"]
Oct 30 05:32:34.400613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:34.400519    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c5ec156-ed7f-4f31-8022-d5fd94e3144c" path="/var/lib/kubelet/pods/3c5ec156-ed7f-4f31-8022-d5fd94e3144c/volumes"
Oct 30 05:32:41.849444 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:41.849371    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b668646-vshtx" podUID="dac2ba5d-8990-47e3-aa65-02ff7500ac01" containerName="registry" containerID="cri-o://bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a" gracePeriod=30
Oct 30 05:32:42.148848 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.148814    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:32:42.283546 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.283514    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dac2ba5d-8990-47e3-aa65-02ff7500ac01-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dac2ba5d-8990-47e3-aa65-02ff7500ac01-ca-trust-extracted\") pod \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") "
Oct 30 05:32:42.283765 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.283565    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-installation-pull-secrets\") pod \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") "
Oct 30 05:32:42.283765 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.283623    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-trusted-ca\") pod \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") "
Oct 30 05:32:42.283765 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.283667    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls\") pod \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") "
Oct 30 05:32:42.283765 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.283721    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-bound-sa-token\") pod \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") "
Oct 30 05:32:42.283992 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.283821    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-kube-api-access-wrchp\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-kube-api-access-wrchp\") pod \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") "
Oct 30 05:32:42.283992 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.283855    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-certificates\") pod \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") "
Oct 30 05:32:42.283992 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.283894    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-image-registry-private-configuration\") pod \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\" (UID: \"dac2ba5d-8990-47e3-aa65-02ff7500ac01\") "
Oct 30 05:32:42.284255 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.284208    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-trusted-ca" pod "dac2ba5d-8990-47e3-aa65-02ff7500ac01" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:42.286853 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.286812    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-installation-pull-secrets" pod "dac2ba5d-8990-47e3-aa65-02ff7500ac01" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:32:42.286960 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.286851    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-image-registry-private-configuration" pod "dac2ba5d-8990-47e3-aa65-02ff7500ac01" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:32:42.287120 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.287089    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-bound-sa-token" pod "dac2ba5d-8990-47e3-aa65-02ff7500ac01" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:32:42.288641 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.288573    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-certificates" pod "dac2ba5d-8990-47e3-aa65-02ff7500ac01" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:42.290217 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.290187    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-kube-api-access-wrchp" pod "dac2ba5d-8990-47e3-aa65-02ff7500ac01" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01"). InnerVolumeSpecName "kube-api-access-wrchp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:32:42.290627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.290568    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls" pod "dac2ba5d-8990-47e3-aa65-02ff7500ac01" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:32:42.292120 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.292099    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dac2ba5d-8990-47e3-aa65-02ff7500ac01-ca-trust-extracted" pod "dac2ba5d-8990-47e3-aa65-02ff7500ac01" (UID: "dac2ba5d-8990-47e3-aa65-02ff7500ac01"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:32:42.385041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.384958    5066 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dac2ba5d-8990-47e3-aa65-02ff7500ac01-ca-trust-extracted\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:42.385041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.384986    5066 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-installation-pull-secrets\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:42.385041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.384997    5066 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-trusted-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:42.385041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.385005    5066 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:42.385041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.385015    5066 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-bound-sa-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:42.385041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.385023    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrchp\" (UniqueName: \"kubernetes.io/projected/dac2ba5d-8990-47e3-aa65-02ff7500ac01-kube-api-access-wrchp\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:42.385041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.385032    5066 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dac2ba5d-8990-47e3-aa65-02ff7500ac01-registry-certificates\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:42.385041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.385044    5066 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dac2ba5d-8990-47e3-aa65-02ff7500ac01-image-registry-private-configuration\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:42.946196 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.946160    5066 generic.go:356] "Generic (PLEG): container finished" podID="dac2ba5d-8990-47e3-aa65-02ff7500ac01" containerID="bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a" exitCode=0
Oct 30 05:32:42.946730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.946218    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b668646-vshtx"
Oct 30 05:32:42.946730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.946245    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b668646-vshtx" event={"ID":"dac2ba5d-8990-47e3-aa65-02ff7500ac01","Type":"ContainerDied","Data":"bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a"}
Oct 30 05:32:42.946730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.946296    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b668646-vshtx" event={"ID":"dac2ba5d-8990-47e3-aa65-02ff7500ac01","Type":"ContainerDied","Data":"fe64e1e0a2e5840cde6a94c13e5c54dac4246fd5e259d0a09cf4543163237a13"}
Oct 30 05:32:42.946730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.946318    5066 scope.go:122] "RemoveContainer" containerID="bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a"
Oct 30 05:32:42.954735 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.954716    5066 scope.go:122] "RemoveContainer" containerID="bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a"
Oct 30 05:32:42.954994 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:32:42.954977    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a\": container with ID starting with bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a not found: ID does not exist" containerID="bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a"
Oct 30 05:32:42.955050 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:42.955002    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a"} err="failed to get container status \"bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a\": rpc error: code = NotFound desc = could not find container \"bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a\": container with ID starting with bedc3b2ae396450a9455797e996ecfd8e2b08b543d4a04e9a45809c93020001a not found: ID does not exist"
Oct 30 05:32:43.019529 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:43.019499    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b668646-vshtx"]
Oct 30 05:32:43.034205 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:43.034179    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b668646-vshtx"]
Oct 30 05:32:44.399614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:44.399567    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dac2ba5d-8990-47e3-aa65-02ff7500ac01" path="/var/lib/kubelet/pods/dac2ba5d-8990-47e3-aa65-02ff7500ac01/volumes"
Oct 30 05:32:45.804632 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:45.804600    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:45.966404 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:45.966377    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:47.163901 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:47.163866    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:47.319324 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:47.319284    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:49.963628 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:49.963573    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6d5ddb484b-xlrs8" podUID="a7792bba-50cd-4e10-8894-3dbcb9204a70" containerName="console" containerID="cri-o://64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842" gracePeriod=15
Oct 30 05:32:50.235543 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.235477    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:32:50.252125 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252099    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-service-ca\") pod \"a7792bba-50cd-4e10-8894-3dbcb9204a70\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") "
Oct 30 05:32:50.252260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252138    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-trusted-ca-bundle\") pod \"a7792bba-50cd-4e10-8894-3dbcb9204a70\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") "
Oct 30 05:32:50.252260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252176    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-oauth-config\") pod \"a7792bba-50cd-4e10-8894-3dbcb9204a70\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") "
Oct 30 05:32:50.252260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252226    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-oauth-serving-cert\") pod \"a7792bba-50cd-4e10-8894-3dbcb9204a70\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") "
Oct 30 05:32:50.252414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252266    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7792bba-50cd-4e10-8894-3dbcb9204a70-kube-api-access-kq6rd\" (UniqueName: \"kubernetes.io/projected/a7792bba-50cd-4e10-8894-3dbcb9204a70-kube-api-access-kq6rd\") pod \"a7792bba-50cd-4e10-8894-3dbcb9204a70\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") "
Oct 30 05:32:50.252414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252333    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-serving-cert\") pod \"a7792bba-50cd-4e10-8894-3dbcb9204a70\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") "
Oct 30 05:32:50.252414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252367    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-config\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-config\") pod \"a7792bba-50cd-4e10-8894-3dbcb9204a70\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") "
Oct 30 05:32:50.252414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252403    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7792bba-50cd-4e10-8894-3dbcb9204a70-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7792bba-50cd-4e10-8894-3dbcb9204a70-tmp\") pod \"a7792bba-50cd-4e10-8894-3dbcb9204a70\" (UID: \"a7792bba-50cd-4e10-8894-3dbcb9204a70\") "
Oct 30 05:32:50.252626 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252536    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-trusted-ca-bundle" pod "a7792bba-50cd-4e10-8894-3dbcb9204a70" (UID: "a7792bba-50cd-4e10-8894-3dbcb9204a70"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:50.252626 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252609    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-service-ca" pod "a7792bba-50cd-4e10-8894-3dbcb9204a70" (UID: "a7792bba-50cd-4e10-8894-3dbcb9204a70"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:50.252779 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252708    5066 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:50.252779 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252726    5066 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-trusted-ca-bundle\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:50.252980 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252958    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-config" pod "a7792bba-50cd-4e10-8894-3dbcb9204a70" (UID: "a7792bba-50cd-4e10-8894-3dbcb9204a70"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:50.252980 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252972    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-oauth-serving-cert" pod "a7792bba-50cd-4e10-8894-3dbcb9204a70" (UID: "a7792bba-50cd-4e10-8894-3dbcb9204a70"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:32:50.253063 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.252966    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7792bba-50cd-4e10-8894-3dbcb9204a70-tmp" pod "a7792bba-50cd-4e10-8894-3dbcb9204a70" (UID: "a7792bba-50cd-4e10-8894-3dbcb9204a70"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:32:50.254902 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.254882    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-oauth-config" pod "a7792bba-50cd-4e10-8894-3dbcb9204a70" (UID: "a7792bba-50cd-4e10-8894-3dbcb9204a70"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:32:50.255150 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.255113    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7792bba-50cd-4e10-8894-3dbcb9204a70-kube-api-access-kq6rd" pod "a7792bba-50cd-4e10-8894-3dbcb9204a70" (UID: "a7792bba-50cd-4e10-8894-3dbcb9204a70"). InnerVolumeSpecName "kube-api-access-kq6rd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:32:50.255257 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.255154    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-serving-cert" pod "a7792bba-50cd-4e10-8894-3dbcb9204a70" (UID: "a7792bba-50cd-4e10-8894-3dbcb9204a70"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:32:50.353580 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.353541    5066 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-oauth-serving-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:50.353580 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.353572    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq6rd\" (UniqueName: \"kubernetes.io/projected/a7792bba-50cd-4e10-8894-3dbcb9204a70-kube-api-access-kq6rd\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:50.353580 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.353582    5066 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-serving-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:50.353823 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.353607    5066 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-config\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:50.353823 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.353617    5066 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7792bba-50cd-4e10-8894-3dbcb9204a70-tmp\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:50.353823 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.353625    5066 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7792bba-50cd-4e10-8894-3dbcb9204a70-console-oauth-config\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:32:50.982250 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.982141    5066 generic.go:356] "Generic (PLEG): container finished" podID="86fadfbe-f000-4b42-a8ff-7b67061959c5" containerID="dc133e01fe17dfe230f1fd4b554e30aa12473775011d8e59ade64628d6602bf6" exitCode=0
Oct 30 05:32:50.982250 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.982226    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-swtbv" event={"ID":"86fadfbe-f000-4b42-a8ff-7b67061959c5","Type":"ContainerDied","Data":"dc133e01fe17dfe230f1fd4b554e30aa12473775011d8e59ade64628d6602bf6"}
Oct 30 05:32:50.982857 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.982295    5066 scope.go:122] "RemoveContainer" containerID="99e07dbd2d5e196e824a4da9856bb03b5e06e0dc1a5ca4ac3d9f6adef617b69d"
Oct 30 05:32:50.983709 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.982984    5066 scope.go:122] "RemoveContainer" containerID="dc133e01fe17dfe230f1fd4b554e30aa12473775011d8e59ade64628d6602bf6"
Oct 30 05:32:50.983709 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:32:50.983248    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"insights-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=insights-operator pod=insights-operator-7c4986d449-swtbv_openshift-insights(86fadfbe-f000-4b42-a8ff-7b67061959c5)\"" pod="openshift-insights/insights-operator-7c4986d449-swtbv" podUID="86fadfbe-f000-4b42-a8ff-7b67061959c5"
Oct 30 05:32:50.985663 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.985642    5066 generic.go:356] "Generic (PLEG): container finished" podID="a7792bba-50cd-4e10-8894-3dbcb9204a70" containerID="64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842" exitCode=0
Oct 30 05:32:50.985750 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.985686    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d5ddb484b-xlrs8" event={"ID":"a7792bba-50cd-4e10-8894-3dbcb9204a70","Type":"ContainerDied","Data":"64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842"}
Oct 30 05:32:50.985750 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.985691    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d5ddb484b-xlrs8"
Oct 30 05:32:50.985750 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:50.985722    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d5ddb484b-xlrs8" event={"ID":"a7792bba-50cd-4e10-8894-3dbcb9204a70","Type":"ContainerDied","Data":"503f81586a69525e8c30f08f9bd47af5a44f2ce3a7b9f8d755ab77dad5b98a46"}
Oct 30 05:32:51.011182 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:51.011155    5066 scope.go:122] "RemoveContainer" containerID="64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842"
Oct 30 05:32:51.019447 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:51.019427    5066 scope.go:122] "RemoveContainer" containerID="64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842"
Oct 30 05:32:51.019763 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:32:51.019743    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842\": container with ID starting with 64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842 not found: ID does not exist" containerID="64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842"
Oct 30 05:32:51.019822 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:51.019772    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842"} err="failed to get container status \"64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842\": rpc error: code = NotFound desc = could not find container \"64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842\": container with ID starting with 64cf5a66b87240e554dd85cccee463e5be56d96c5b627494c3a2081c9bf13842 not found: ID does not exist"
Oct 30 05:32:51.073233 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:51.073191    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d5ddb484b-xlrs8"]
Oct 30 05:32:51.077074 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:51.077048    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d5ddb484b-xlrs8"]
Oct 30 05:32:52.399758 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:52.399726    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7792bba-50cd-4e10-8894-3dbcb9204a70" path="/var/lib/kubelet/pods/a7792bba-50cd-4e10-8894-3dbcb9204a70/volumes"
Oct 30 05:32:54.100680 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.100638    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-84fb794766-rps98"]
Oct 30 05:32:54.101278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101080    5066 cpu_manager.go:414] "containerMap: removing container" podUID="dac2ba5d-8990-47e3-aa65-02ff7500ac01" containerName="registry"
Oct 30 05:32:54.101278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101096    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dac2ba5d-8990-47e3-aa65-02ff7500ac01" containerName="registry"
Oct 30 05:32:54.101278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101107    5066 cpu_manager.go:414] "containerMap: removing container" podUID="3c5ec156-ed7f-4f31-8022-d5fd94e3144c" containerName="console"
Oct 30 05:32:54.101278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101115    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c5ec156-ed7f-4f31-8022-d5fd94e3144c" containerName="console"
Oct 30 05:32:54.101278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101126    5066 cpu_manager.go:414] "containerMap: removing container" podUID="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" containerName="registry"
Oct 30 05:32:54.101278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101134    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" containerName="registry"
Oct 30 05:32:54.101278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101155    5066 cpu_manager.go:414] "containerMap: removing container" podUID="f74041a6-7409-439c-9ecb-4bc8dcc2006e" containerName="insights-gathering"
Oct 30 05:32:54.101278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101161    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f74041a6-7409-439c-9ecb-4bc8dcc2006e" containerName="insights-gathering"
Oct 30 05:32:54.101278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101185    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a7792bba-50cd-4e10-8894-3dbcb9204a70" containerName="console"
Oct 30 05:32:54.101278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101191    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7792bba-50cd-4e10-8894-3dbcb9204a70" containerName="console"
Oct 30 05:32:54.101737 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101299    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="dac2ba5d-8990-47e3-aa65-02ff7500ac01" containerName="registry"
Oct 30 05:32:54.101737 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101309    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c5ec156-ed7f-4f31-8022-d5fd94e3144c" containerName="console"
Oct 30 05:32:54.101737 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101318    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="e877e7c6-b2f6-4e79-ae22-2e04dcd3e76a" containerName="registry"
Oct 30 05:32:54.101737 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101328    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="f74041a6-7409-439c-9ecb-4bc8dcc2006e" containerName="insights-gathering"
Oct 30 05:32:54.101737 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.101337    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7792bba-50cd-4e10-8894-3dbcb9204a70" containerName="console"
Oct 30 05:32:54.130707 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.130674    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84fb794766-rps98"]
Oct 30 05:32:54.131125 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.131103    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.188748 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.188721    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-oauth-config\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.188901 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.188764    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60c4df8-661a-4f69-ae7a-c7fd760003f8-tmp\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.188901 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.188875    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-oauth-serving-cert\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.188979 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.188912    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-trusted-ca-bundle\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.189015 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.189003    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpx72\" (UniqueName: \"kubernetes.io/projected/b60c4df8-661a-4f69-ae7a-c7fd760003f8-kube-api-access-zpx72\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.189048 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.189020    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-service-ca\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.189080 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.189051    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-serving-cert\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.189140 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.189125    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-config\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.290229 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.290192    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-trusted-ca-bundle\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.290400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.290236    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpx72\" (UniqueName: \"kubernetes.io/projected/b60c4df8-661a-4f69-ae7a-c7fd760003f8-kube-api-access-zpx72\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.290400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.290253    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-service-ca\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.290400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.290277    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-serving-cert\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.290400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.290389    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-config\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.290635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.290423    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-oauth-config\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.290635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.290468    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60c4df8-661a-4f69-ae7a-c7fd760003f8-tmp\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.290635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.290516    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-oauth-serving-cert\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.290981 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.290956    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60c4df8-661a-4f69-ae7a-c7fd760003f8-tmp\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.291069 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.291058    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-service-ca\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.291175 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.291145    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-config\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.291425 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.291402    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-oauth-serving-cert\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.291557 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.291534    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-trusted-ca-bundle\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.293052 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.293033    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-serving-cert\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.293137 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.293082    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-oauth-config\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.293298 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.293252    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:54.315825 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.315801    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpx72\" (UniqueName: \"kubernetes.io/projected/b60c4df8-661a-4f69-ae7a-c7fd760003f8-kube-api-access-zpx72\") pod \"console-84fb794766-rps98\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") " pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.458347 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.458306    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:54.486313 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.486283    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:32:54.660343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.660310    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:54.829534 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:54.829509    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84fb794766-rps98"]
Oct 30 05:32:54.831445 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:32:54.831420    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60c4df8_661a_4f69_ae7a_c7fd760003f8.slice/crio-c0ae501b5ae04c33ccc90b0fd61a38fd6b1f28c7e67816a52ad4692166d86d4c WatchSource:0}: Error finding container c0ae501b5ae04c33ccc90b0fd61a38fd6b1f28c7e67816a52ad4692166d86d4c: Status 404 returned error can't find the container with id c0ae501b5ae04c33ccc90b0fd61a38fd6b1f28c7e67816a52ad4692166d86d4c
Oct 30 05:32:55.009797 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:55.009754    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84fb794766-rps98" event={"ID":"b60c4df8-661a-4f69-ae7a-c7fd760003f8","Type":"ContainerStarted","Data":"de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61"}
Oct 30 05:32:55.010003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:55.009817    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84fb794766-rps98" event={"ID":"b60c4df8-661a-4f69-ae7a-c7fd760003f8","Type":"ContainerStarted","Data":"c0ae501b5ae04c33ccc90b0fd61a38fd6b1f28c7e67816a52ad4692166d86d4c"}
Oct 30 05:32:55.057393 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:55.057343    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-84fb794766-rps98" podStartSLOduration=1.057328611 podStartE2EDuration="1.057328611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:32:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:32:55.056635971 +0000 UTC m=+78.801921303" watchObservedRunningTime="2026-10-30 05:32:55.057328611 +0000 UTC m=+78.802613944"
Oct 30 05:32:58.598445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:58.598413    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:32:58.758692 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:32:58.758654    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:33:02.550321 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:33:02.550283    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:33:04.486724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:04.486687    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:33:04.487124 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:04.486743    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:33:04.491651 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:04.491631    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:33:05.054859 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:05.054831    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:33:05.132160 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:05.132131    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b4cccf476-wkhc5"]
Oct 30 05:33:05.394484 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:05.394406    5066 scope.go:122] "RemoveContainer" containerID="dc133e01fe17dfe230f1fd4b554e30aa12473775011d8e59ade64628d6602bf6"
Oct 30 05:33:06.057283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.057248    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-swtbv" event={"ID":"86fadfbe-f000-4b42-a8ff-7b67061959c5","Type":"ContainerStarted","Data":"53a3fa01da5d27fcf308bf66033a87ab203bae7113bb0a8b92e29f838e6fdd8a"}
Oct 30 05:33:06.142814 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.142782    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-gzmfz-hfdrp"]
Oct 30 05:33:06.143037 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.143012    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-gzmfz-hfdrp" podUID="a9ccf6bc-15a2-4779-bb91-3df7ed70d46a" containerName="insights-gathering" containerID="cri-o://06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3" gracePeriod=30
Oct 30 05:33:06.392206 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.392175    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:33:06.497892 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.497858    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-archives-path\") pod \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") "
Oct 30 05:33:06.497892 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.497892    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-service-ca-bundle\") pod \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") "
Oct 30 05:33:06.498146 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.497912    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-kube-api-access-8p6r7\" (UniqueName: \"kubernetes.io/projected/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-kube-api-access-8p6r7\") pod \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") "
Oct 30 05:33:06.498146 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.497934    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-trusted-ca-bundle\") pod \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\" (UID: \"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a\") "
Oct 30 05:33:06.498261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.498211    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-archives-path" pod "a9ccf6bc-15a2-4779-bb91-3df7ed70d46a" (UID: "a9ccf6bc-15a2-4779-bb91-3df7ed70d46a"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:33:06.498314 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.498258    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-service-ca-bundle" pod "a9ccf6bc-15a2-4779-bb91-3df7ed70d46a" (UID: "a9ccf6bc-15a2-4779-bb91-3df7ed70d46a"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:33:06.498414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.498376    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-trusted-ca-bundle" pod "a9ccf6bc-15a2-4779-bb91-3df7ed70d46a" (UID: "a9ccf6bc-15a2-4779-bb91-3df7ed70d46a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:33:06.498755 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.498727    5066 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-archives-path\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:33:06.498755 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.498757    5066 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-service-ca-bundle\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:33:06.498904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.498772    5066 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-trusted-ca-bundle\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:33:06.500305 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.500288    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-kube-api-access-8p6r7" pod "a9ccf6bc-15a2-4779-bb91-3df7ed70d46a" (UID: "a9ccf6bc-15a2-4779-bb91-3df7ed70d46a"). InnerVolumeSpecName "kube-api-access-8p6r7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:33:06.599955 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:06.599876    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p6r7\" (UniqueName: \"kubernetes.io/projected/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a-kube-api-access-8p6r7\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:33:07.062198 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:07.062159    5066 generic.go:356] "Generic (PLEG): container finished" podID="a9ccf6bc-15a2-4779-bb91-3df7ed70d46a" containerID="06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3" exitCode=2
Oct 30 05:33:07.062783 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:07.062210    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-gzmfz-hfdrp"
Oct 30 05:33:07.062783 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:07.062248    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-gzmfz-hfdrp" event={"ID":"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a","Type":"ContainerDied","Data":"06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3"}
Oct 30 05:33:07.062783 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:07.062317    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-gzmfz-hfdrp" event={"ID":"a9ccf6bc-15a2-4779-bb91-3df7ed70d46a","Type":"ContainerDied","Data":"d60b3a92890631ce6980089ae4f16c2db93979ba6feef71fe862b718a013be82"}
Oct 30 05:33:07.062783 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:07.062346    5066 scope.go:122] "RemoveContainer" containerID="06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3"
Oct 30 05:33:07.070861 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:07.070844    5066 scope.go:122] "RemoveContainer" containerID="06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3"
Oct 30 05:33:07.071113 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:33:07.071095    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3\": container with ID starting with 06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3 not found: ID does not exist" containerID="06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3"
Oct 30 05:33:07.071154 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:07.071123    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3"} err="failed to get container status \"06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3\": rpc error: code = NotFound desc = could not find container \"06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3\": container with ID starting with 06197eb6e707d938dda6f2cfd7575e3534c1685c6d942b5fa78859420e92d0a3 not found: ID does not exist"
Oct 30 05:33:07.119749 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:07.119725    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-gzmfz-hfdrp"]
Oct 30 05:33:07.123565 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:07.123547    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-gzmfz-hfdrp"]
Oct 30 05:33:08.400049 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:08.400018    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9ccf6bc-15a2-4779-bb91-3df7ed70d46a" path="/var/lib/kubelet/pods/a9ccf6bc-15a2-4779-bb91-3df7ed70d46a/volumes"
Oct 30 05:33:25.600231 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:25.600198    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:33:30.152261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.152195    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5b4cccf476-wkhc5" podUID="a10f6321-4f07-4e27-bf4a-5113e0746b01" containerName="console" containerID="cri-o://4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0" gracePeriod=15
Oct 30 05:33:30.411948 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.411925    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:33:30.510610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.510557    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-oauth-config\") pod \"a10f6321-4f07-4e27-bf4a-5113e0746b01\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") "
Oct 30 05:33:30.510610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.510614    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a10f6321-4f07-4e27-bf4a-5113e0746b01-kube-api-access-rtrfh\" (UniqueName: \"kubernetes.io/projected/a10f6321-4f07-4e27-bf4a-5113e0746b01-kube-api-access-rtrfh\") pod \"a10f6321-4f07-4e27-bf4a-5113e0746b01\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") "
Oct 30 05:33:30.510841 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.510652    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-oauth-serving-cert\") pod \"a10f6321-4f07-4e27-bf4a-5113e0746b01\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") "
Oct 30 05:33:30.510841 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.510674    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a10f6321-4f07-4e27-bf4a-5113e0746b01-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a10f6321-4f07-4e27-bf4a-5113e0746b01-tmp\") pod \"a10f6321-4f07-4e27-bf4a-5113e0746b01\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") "
Oct 30 05:33:30.510841 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.510695    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-service-ca\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-service-ca\") pod \"a10f6321-4f07-4e27-bf4a-5113e0746b01\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") "
Oct 30 05:33:30.510841 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.510723    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-config\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-config\") pod \"a10f6321-4f07-4e27-bf4a-5113e0746b01\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") "
Oct 30 05:33:30.510841 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.510780    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-trusted-ca-bundle\") pod \"a10f6321-4f07-4e27-bf4a-5113e0746b01\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") "
Oct 30 05:33:30.510841 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.510807    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-serving-cert\") pod \"a10f6321-4f07-4e27-bf4a-5113e0746b01\" (UID: \"a10f6321-4f07-4e27-bf4a-5113e0746b01\") "
Oct 30 05:33:30.511424 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.511046    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10f6321-4f07-4e27-bf4a-5113e0746b01-tmp" pod "a10f6321-4f07-4e27-bf4a-5113e0746b01" (UID: "a10f6321-4f07-4e27-bf4a-5113e0746b01"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:33:30.511424 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.511184    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-oauth-serving-cert" pod "a10f6321-4f07-4e27-bf4a-5113e0746b01" (UID: "a10f6321-4f07-4e27-bf4a-5113e0746b01"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:33:30.511424 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.511216    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-service-ca" pod "a10f6321-4f07-4e27-bf4a-5113e0746b01" (UID: "a10f6321-4f07-4e27-bf4a-5113e0746b01"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:33:30.511424 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.511281    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-config" pod "a10f6321-4f07-4e27-bf4a-5113e0746b01" (UID: "a10f6321-4f07-4e27-bf4a-5113e0746b01"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:33:30.511570 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.511427    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-trusted-ca-bundle" pod "a10f6321-4f07-4e27-bf4a-5113e0746b01" (UID: "a10f6321-4f07-4e27-bf4a-5113e0746b01"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:33:30.512953 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.512934    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a10f6321-4f07-4e27-bf4a-5113e0746b01-kube-api-access-rtrfh" pod "a10f6321-4f07-4e27-bf4a-5113e0746b01" (UID: "a10f6321-4f07-4e27-bf4a-5113e0746b01"). InnerVolumeSpecName "kube-api-access-rtrfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:33:30.513018 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.512964    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-oauth-config" pod "a10f6321-4f07-4e27-bf4a-5113e0746b01" (UID: "a10f6321-4f07-4e27-bf4a-5113e0746b01"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:33:30.513123 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.513108    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-serving-cert" pod "a10f6321-4f07-4e27-bf4a-5113e0746b01" (UID: "a10f6321-4f07-4e27-bf4a-5113e0746b01"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:33:30.611720 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.611686    5066 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-oauth-serving-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:33:30.611720 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.611713    5066 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a10f6321-4f07-4e27-bf4a-5113e0746b01-tmp\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:33:30.611720 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.611726    5066 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:33:30.611955 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.611735    5066 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-config\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:33:30.611955 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.611743    5066 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a10f6321-4f07-4e27-bf4a-5113e0746b01-trusted-ca-bundle\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:33:30.611955 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.611752    5066 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-serving-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:33:30.611955 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.611760    5066 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a10f6321-4f07-4e27-bf4a-5113e0746b01-console-oauth-config\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:33:30.611955 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:30.611768    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtrfh\" (UniqueName: \"kubernetes.io/projected/a10f6321-4f07-4e27-bf4a-5113e0746b01-kube-api-access-rtrfh\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:33:31.170971 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:31.170940    5066 generic.go:356] "Generic (PLEG): container finished" podID="a10f6321-4f07-4e27-bf4a-5113e0746b01" containerID="4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0" exitCode=0
Oct 30 05:33:31.171410 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:31.170993    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4cccf476-wkhc5"
Oct 30 05:33:31.171410 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:31.171031    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4cccf476-wkhc5" event={"ID":"a10f6321-4f07-4e27-bf4a-5113e0746b01","Type":"ContainerDied","Data":"4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0"}
Oct 30 05:33:31.171410 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:31.171097    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4cccf476-wkhc5" event={"ID":"a10f6321-4f07-4e27-bf4a-5113e0746b01","Type":"ContainerDied","Data":"a5b6aa547d642b1fd43db0bf12117ff23f69826d2cd17e2ed2c5e2efdcb0773c"}
Oct 30 05:33:31.171410 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:31.171125    5066 scope.go:122] "RemoveContainer" containerID="4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0"
Oct 30 05:33:31.183419 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:31.183401    5066 scope.go:122] "RemoveContainer" containerID="4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0"
Oct 30 05:33:31.183748 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:33:31.183722    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0\": container with ID starting with 4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0 not found: ID does not exist" containerID="4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0"
Oct 30 05:33:31.183857 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:31.183758    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0"} err="failed to get container status \"4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0\": rpc error: code = NotFound desc = could not find container \"4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0\": container with ID starting with 4eb40e6d2a6eaf94d1a946e3970efa3dfe81d31a74bf5ce0162b0d4d60cf8ea0 not found: ID does not exist"
Oct 30 05:33:31.229705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:31.229678    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b4cccf476-wkhc5"]
Oct 30 05:33:31.244264 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:31.244236    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b4cccf476-wkhc5"]
Oct 30 05:33:31.572314 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:31.572226    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:33:31.731244 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:31.731213    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:33:32.400140 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:32.400110    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a10f6321-4f07-4e27-bf4a-5113e0746b01" path="/var/lib/kubelet/pods/a10f6321-4f07-4e27-bf4a-5113e0746b01/volumes"
Oct 30 05:33:32.550434 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:33:32.550401    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:33:38.763197 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.763168    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-84ff984b45-g6jd8"]
Oct 30 05:33:38.763799 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.763698    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a9ccf6bc-15a2-4779-bb91-3df7ed70d46a" containerName="insights-gathering"
Oct 30 05:33:38.763799 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.763717    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9ccf6bc-15a2-4779-bb91-3df7ed70d46a" containerName="insights-gathering"
Oct 30 05:33:38.763799 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.763764    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a10f6321-4f07-4e27-bf4a-5113e0746b01" containerName="console"
Oct 30 05:33:38.763799 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.763772    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a10f6321-4f07-4e27-bf4a-5113e0746b01" containerName="console"
Oct 30 05:33:38.764002 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.763851    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9ccf6bc-15a2-4779-bb91-3df7ed70d46a" containerName="insights-gathering"
Oct 30 05:33:38.764002 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.763869    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a10f6321-4f07-4e27-bf4a-5113e0746b01" containerName="console"
Oct 30 05:33:38.766710 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.766690    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.789450 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.789423    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84ff984b45-g6jd8"]
Oct 30 05:33:38.867254 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.867218    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/945444af-d32f-4647-a40d-7d68802a9b9c-console-serving-cert\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.867432 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.867261    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/945444af-d32f-4647-a40d-7d68802a9b9c-trusted-ca-bundle\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.867432 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.867284    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/945444af-d32f-4647-a40d-7d68802a9b9c-tmp\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.867432 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.867383    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/945444af-d32f-4647-a40d-7d68802a9b9c-oauth-serving-cert\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.867432 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.867424    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/945444af-d32f-4647-a40d-7d68802a9b9c-console-oauth-config\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.867670 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.867518    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/945444af-d32f-4647-a40d-7d68802a9b9c-console-config\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.867670 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.867559    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/945444af-d32f-4647-a40d-7d68802a9b9c-service-ca\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.867670 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.867619    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9btwp\" (UniqueName: \"kubernetes.io/projected/945444af-d32f-4647-a40d-7d68802a9b9c-kube-api-access-9btwp\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.967982 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.967945    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9btwp\" (UniqueName: \"kubernetes.io/projected/945444af-d32f-4647-a40d-7d68802a9b9c-kube-api-access-9btwp\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.968169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.967995    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/945444af-d32f-4647-a40d-7d68802a9b9c-console-serving-cert\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.968169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.968026    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/945444af-d32f-4647-a40d-7d68802a9b9c-trusted-ca-bundle\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.968169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.968050    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/945444af-d32f-4647-a40d-7d68802a9b9c-tmp\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.968169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.968074    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/945444af-d32f-4647-a40d-7d68802a9b9c-oauth-serving-cert\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.968169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.968118    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/945444af-d32f-4647-a40d-7d68802a9b9c-console-oauth-config\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.968169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.968158    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/945444af-d32f-4647-a40d-7d68802a9b9c-console-config\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.968439 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.968192    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/945444af-d32f-4647-a40d-7d68802a9b9c-service-ca\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.968546 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.968517    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/945444af-d32f-4647-a40d-7d68802a9b9c-tmp\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.969008 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.968980    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/945444af-d32f-4647-a40d-7d68802a9b9c-oauth-serving-cert\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.969126 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.969033    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/945444af-d32f-4647-a40d-7d68802a9b9c-service-ca\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.969126 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.969059    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/945444af-d32f-4647-a40d-7d68802a9b9c-trusted-ca-bundle\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.969245 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.969221    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/945444af-d32f-4647-a40d-7d68802a9b9c-console-config\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.970379 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.970358    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:33:38.970814 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.970787    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/945444af-d32f-4647-a40d-7d68802a9b9c-console-oauth-config\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.971016 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.970998    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/945444af-d32f-4647-a40d-7d68802a9b9c-console-serving-cert\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:38.993681 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:38.993654    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9btwp\" (UniqueName: \"kubernetes.io/projected/945444af-d32f-4647-a40d-7d68802a9b9c-kube-api-access-9btwp\") pod \"console-84ff984b45-g6jd8\" (UID: \"945444af-d32f-4647-a40d-7d68802a9b9c\") " pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:39.076953 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:39.076870    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:39.131478 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:39.131448    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:33:39.376800 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:39.376780    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84ff984b45-g6jd8"]
Oct 30 05:33:39.377957 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:33:39.377930    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945444af_d32f_4647_a40d_7d68802a9b9c.slice/crio-6361a7ad6bf0dad45d98b63e08f4978c316950ab58ec19f0cfceda08257246f9 WatchSource:0}: Error finding container 6361a7ad6bf0dad45d98b63e08f4978c316950ab58ec19f0cfceda08257246f9: Status 404 returned error can't find the container with id 6361a7ad6bf0dad45d98b63e08f4978c316950ab58ec19f0cfceda08257246f9
Oct 30 05:33:40.211903 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:40.211863    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84ff984b45-g6jd8" event={"ID":"945444af-d32f-4647-a40d-7d68802a9b9c","Type":"ContainerStarted","Data":"e42e134675e149ee6dd665d997a3c128babca55ca841cf8922c10baa893f9447"}
Oct 30 05:33:40.211903 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:40.211910    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84ff984b45-g6jd8" event={"ID":"945444af-d32f-4647-a40d-7d68802a9b9c","Type":"ContainerStarted","Data":"6361a7ad6bf0dad45d98b63e08f4978c316950ab58ec19f0cfceda08257246f9"}
Oct 30 05:33:40.274074 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:40.274020    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-84ff984b45-g6jd8" podStartSLOduration=2.274004829 podStartE2EDuration="2.274004829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:33:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:33:40.258897223 +0000 UTC m=+124.004182555" watchObservedRunningTime="2026-10-30 05:33:40.274004829 +0000 UTC m=+124.019290160"
Oct 30 05:33:40.592114 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:40.592035    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:33:40.755353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:40.755316    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:33:44.303321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:44.303287    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:33:44.455424 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:44.455381    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:33:47.210928 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:47.210845    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:33:47.367622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:47.367564    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:33:49.077488 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:49.077459    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:49.077855 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:49.077770    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:49.082571 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:49.082551    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:49.254227 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:49.254202    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84ff984b45-g6jd8"
Oct 30 05:33:49.330969 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:33:49.330900    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84fb794766-rps98"]
Oct 30 05:34:02.550672 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:34:02.550636    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:34:14.355053 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.354993    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-84fb794766-rps98" podUID="b60c4df8-661a-4f69-ae7a-c7fd760003f8" containerName="console" containerID="cri-o://de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61" gracePeriod=15
Oct 30 05:34:14.601643 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.601617    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:34:14.672875 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.672841    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-oauth-config\") pod \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") "
Oct 30 05:34:14.673041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.672891    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b60c4df8-661a-4f69-ae7a-c7fd760003f8-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60c4df8-661a-4f69-ae7a-c7fd760003f8-tmp\") pod \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") "
Oct 30 05:34:14.673041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.672981    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-trusted-ca-bundle\") pod \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") "
Oct 30 05:34:14.673041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.673010    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-oauth-serving-cert\") pod \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") "
Oct 30 05:34:14.673216 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.673068    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-service-ca\") pod \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") "
Oct 30 05:34:14.673216 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.673097    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-serving-cert\") pod \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") "
Oct 30 05:34:14.673332 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.673217    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b60c4df8-661a-4f69-ae7a-c7fd760003f8-tmp" pod "b60c4df8-661a-4f69-ae7a-c7fd760003f8" (UID: "b60c4df8-661a-4f69-ae7a-c7fd760003f8"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:34:14.673386 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.673352    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-config\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-config\") pod \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") "
Oct 30 05:34:14.673466 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.673436    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b60c4df8-661a-4f69-ae7a-c7fd760003f8-kube-api-access-zpx72\" (UniqueName: \"kubernetes.io/projected/b60c4df8-661a-4f69-ae7a-c7fd760003f8-kube-api-access-zpx72\") pod \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\" (UID: \"b60c4df8-661a-4f69-ae7a-c7fd760003f8\") "
Oct 30 05:34:14.673581 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.673432    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-trusted-ca-bundle" pod "b60c4df8-661a-4f69-ae7a-c7fd760003f8" (UID: "b60c4df8-661a-4f69-ae7a-c7fd760003f8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:34:14.673729 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.673481    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-service-ca" pod "b60c4df8-661a-4f69-ae7a-c7fd760003f8" (UID: "b60c4df8-661a-4f69-ae7a-c7fd760003f8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:34:14.673729 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.673702    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-config" pod "b60c4df8-661a-4f69-ae7a-c7fd760003f8" (UID: "b60c4df8-661a-4f69-ae7a-c7fd760003f8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:34:14.673856 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.673728    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-oauth-serving-cert" pod "b60c4df8-661a-4f69-ae7a-c7fd760003f8" (UID: "b60c4df8-661a-4f69-ae7a-c7fd760003f8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:34:14.674165 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.674137    5066 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60c4df8-661a-4f69-ae7a-c7fd760003f8-tmp\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:34:14.674259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.674171    5066 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-trusted-ca-bundle\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:34:14.674259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.674185    5066 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-oauth-serving-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:34:14.674259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.674200    5066 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:34:14.674259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.674213    5066 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-config\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:34:14.675393 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.675364    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-oauth-config" pod "b60c4df8-661a-4f69-ae7a-c7fd760003f8" (UID: "b60c4df8-661a-4f69-ae7a-c7fd760003f8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:34:14.675533 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.675517    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b60c4df8-661a-4f69-ae7a-c7fd760003f8-kube-api-access-zpx72" pod "b60c4df8-661a-4f69-ae7a-c7fd760003f8" (UID: "b60c4df8-661a-4f69-ae7a-c7fd760003f8"). InnerVolumeSpecName "kube-api-access-zpx72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:34:14.675713 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.675697    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-serving-cert" pod "b60c4df8-661a-4f69-ae7a-c7fd760003f8" (UID: "b60c4df8-661a-4f69-ae7a-c7fd760003f8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:34:14.775400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.775367    5066 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-serving-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:34:14.775400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.775395    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpx72\" (UniqueName: \"kubernetes.io/projected/b60c4df8-661a-4f69-ae7a-c7fd760003f8-kube-api-access-zpx72\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:34:14.775400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:14.775404    5066 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b60c4df8-661a-4f69-ae7a-c7fd760003f8-console-oauth-config\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:34:15.360896 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:15.360859    5066 generic.go:356] "Generic (PLEG): container finished" podID="b60c4df8-661a-4f69-ae7a-c7fd760003f8" containerID="de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61" exitCode=0
Oct 30 05:34:15.361311 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:15.360910    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84fb794766-rps98"
Oct 30 05:34:15.361311 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:15.360914    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84fb794766-rps98" event={"ID":"b60c4df8-661a-4f69-ae7a-c7fd760003f8","Type":"ContainerDied","Data":"de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61"}
Oct 30 05:34:15.361311 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:15.360963    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84fb794766-rps98" event={"ID":"b60c4df8-661a-4f69-ae7a-c7fd760003f8","Type":"ContainerDied","Data":"c0ae501b5ae04c33ccc90b0fd61a38fd6b1f28c7e67816a52ad4692166d86d4c"}
Oct 30 05:34:15.361311 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:15.360988    5066 scope.go:122] "RemoveContainer" containerID="de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61"
Oct 30 05:34:15.369794 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:15.369776    5066 scope.go:122] "RemoveContainer" containerID="de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61"
Oct 30 05:34:15.370094 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:34:15.370075    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61\": container with ID starting with de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61 not found: ID does not exist" containerID="de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61"
Oct 30 05:34:15.370138 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:15.370104    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61"} err="failed to get container status \"de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61\": rpc error: code = NotFound desc = could not find container \"de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61\": container with ID starting with de9a36e5a0d0c334326fdaa5b85fd78d51e537eb58006888bb2544c3a2d22e61 not found: ID does not exist"
Oct 30 05:34:15.418369 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:15.418340    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84fb794766-rps98"]
Oct 30 05:34:15.433180 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:15.433149    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84fb794766-rps98"]
Oct 30 05:34:16.399718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:16.399687    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b60c4df8-661a-4f69-ae7a-c7fd760003f8" path="/var/lib/kubelet/pods/b60c4df8-661a-4f69-ae7a-c7fd760003f8/volumes"
Oct 30 05:34:25.720527 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:25.720482    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:34:32.006957 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:34:32.006910    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:34:32.551275 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:34:32.551240    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:35:02.550576 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:35:02.550531    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:35:25.943358 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:35:25.943268    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:35:32.549642 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:35:32.549601    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:36:02.550168 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:36:02.550131    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:36:14.424789 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:14.424758    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:16.226245 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:16.226213    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:25.968976 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:25.968940    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:32.550663 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:36:32.550624    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:36:36.416324 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:36.416282    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:36:36.418340 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:36.418318    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:36:36.512037 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:36.512009    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:37.897677 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:37.897638    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:39.935745 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:39.935706    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:40.116014 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:40.115981    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:41.625937 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:41.625907    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:41.795344 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:41.795305    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:42.272002 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:42.271967    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:42.471916 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:42.471885    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:44.018122 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:44.018088    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:44.460000 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:44.459964    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:44.649327 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:44.649295    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:44.830260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:44.830178    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:46.712711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:46.712634    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:46.877430 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:46.877399    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:51.249509 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:51.249476    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:54.812311 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:54.812273    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:36:54.986671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:36:54.986629    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:00.767017 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:00.766968    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:00.925539 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:00.925486    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:02.522962 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:02.522915    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:02.550243 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:37:02.550213    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:37:05.432768 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:05.432734    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:05.612174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:05.612139    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:12.602444 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:12.602407    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:16.959171 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:16.959141    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:17.131644 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:17.131610    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:17.792281 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:17.792245    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:17.918434 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:17.918399    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"]
Oct 30 05:37:17.918825 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:17.918810    5066 cpu_manager.go:414] "containerMap: removing container" podUID="b60c4df8-661a-4f69-ae7a-c7fd760003f8" containerName="console"
Oct 30 05:37:17.918871 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:17.918829    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b60c4df8-661a-4f69-ae7a-c7fd760003f8" containerName="console"
Oct 30 05:37:17.918904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:17.918883    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="b60c4df8-661a-4f69-ae7a-c7fd760003f8" containerName="console"
Oct 30 05:37:17.921956 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:17.921939    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:17.940627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:17.940581    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"]
Oct 30 05:37:18.007481 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.007445    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\""
Oct 30 05:37:18.007481 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.007475    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 05:37:18.007481 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.007485    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 05:37:18.008041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.007644    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dmsd5\""
Oct 30 05:37:18.008041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.007689    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 05:37:18.008041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.007760    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 05:37:18.060327 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.060244    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/26dbd02d-2395-4eec-b0d0-d46ba647d76f-metrics-cert\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.060327 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.060282    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/26dbd02d-2395-4eec-b0d0-d46ba647d76f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.060519 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.060381    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/26dbd02d-2395-4eec-b0d0-d46ba647d76f-manifests\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.060519 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.060460    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gjd7\" (UniqueName: \"kubernetes.io/projected/26dbd02d-2395-4eec-b0d0-d46ba647d76f-kube-api-access-6gjd7\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.060519 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.060497    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26dbd02d-2395-4eec-b0d0-d46ba647d76f-tmp\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.060519 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.060518    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26dbd02d-2395-4eec-b0d0-d46ba647d76f-webhook-cert\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.086276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.086245    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:18.161929 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.161891    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/26dbd02d-2395-4eec-b0d0-d46ba647d76f-manifests\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.161929 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.161934    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gjd7\" (UniqueName: \"kubernetes.io/projected/26dbd02d-2395-4eec-b0d0-d46ba647d76f-kube-api-access-6gjd7\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.162166 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.162144    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26dbd02d-2395-4eec-b0d0-d46ba647d76f-tmp\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.162228 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.162191    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26dbd02d-2395-4eec-b0d0-d46ba647d76f-webhook-cert\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.162301 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.162281    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/26dbd02d-2395-4eec-b0d0-d46ba647d76f-manifests\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.162363 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.162285    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/26dbd02d-2395-4eec-b0d0-d46ba647d76f-metrics-cert\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.162418 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.162357    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/26dbd02d-2395-4eec-b0d0-d46ba647d76f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.164657 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.164632    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26dbd02d-2395-4eec-b0d0-d46ba647d76f-tmp\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.164922 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.164906    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/26dbd02d-2395-4eec-b0d0-d46ba647d76f-metrics-cert\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.164969 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.164913    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/26dbd02d-2395-4eec-b0d0-d46ba647d76f-webhook-cert\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.164969 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.164947    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/26dbd02d-2395-4eec-b0d0-d46ba647d76f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.187270 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.187244    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gjd7\" (UniqueName: \"kubernetes.io/projected/26dbd02d-2395-4eec-b0d0-d46ba647d76f-kube-api-access-6gjd7\") pod \"opendatahub-operator-controller-manager-f7d897f7-4c5zt\" (UID: \"26dbd02d-2395-4eec-b0d0-d46ba647d76f\") " pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.237312 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.237278    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:18.271643 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.271601    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:18.478008 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.477977    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:18.601892 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.601866    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"]
Oct 30 05:37:18.603555 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:37:18.603535    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26dbd02d_2395_4eec_b0d0_d46ba647d76f.slice/crio-5f545d89425fa5689b9ff165b56ce8615305e78389f5e608bd6535d2d168c854 WatchSource:0}: Error finding container 5f545d89425fa5689b9ff165b56ce8615305e78389f5e608bd6535d2d168c854: Status 404 returned error can't find the container with id 5f545d89425fa5689b9ff165b56ce8615305e78389f5e608bd6535d2d168c854
Oct 30 05:37:18.605172 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:18.605153    5066 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 05:37:19.115782 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:19.115734    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt" event={"ID":"26dbd02d-2395-4eec-b0d0-d46ba647d76f","Type":"ContainerStarted","Data":"5f545d89425fa5689b9ff165b56ce8615305e78389f5e608bd6535d2d168c854"}
Oct 30 05:37:22.131306 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:22.131212    5066 generic.go:356] "Generic (PLEG): container finished" podID="26dbd02d-2395-4eec-b0d0-d46ba647d76f" containerID="d9cfd04b651a8f1bb76091b625c43155285a5d91c2df86414fbfb2d6835aeb73" exitCode=0
Oct 30 05:37:22.131740 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:22.131298    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt" event={"ID":"26dbd02d-2395-4eec-b0d0-d46ba647d76f","Type":"ContainerDied","Data":"d9cfd04b651a8f1bb76091b625c43155285a5d91c2df86414fbfb2d6835aeb73"}
Oct 30 05:37:23.137031 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:23.136994    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt" event={"ID":"26dbd02d-2395-4eec-b0d0-d46ba647d76f","Type":"ContainerStarted","Data":"8d15702cd000df78c602ba926fed653bdee57f0e7b15347afba35fe4dd787167"}
Oct 30 05:37:23.137426 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:23.137148    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:23.233259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:23.233207    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt" podStartSLOduration=2.800933101 podStartE2EDuration="6.23319231s" totalImagesPullingTime="2.992217449s" totalInitContainerRuntime="440.04176ms" isStatefulPod=false podCreationTimestamp="2026-10-30 05:37:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:37:23.2179975 +0000 UTC m=+346.963282833" watchObservedRunningTime="2026-10-30 05:37:23.23319231 +0000 UTC m=+346.978477641"
Oct 30 05:37:26.092009 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:26.091976    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:27.031084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:27.031050    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:27.187149 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:27.187118    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:32.550066 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:37:32.550028    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:37:34.144301 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:34.144277    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f7d897f7-4c5zt"
Oct 30 05:37:35.526810 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.526779    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"]
Oct 30 05:37:35.533164 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.533141    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.551244 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.551218    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"]
Oct 30 05:37:35.597800 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.597776    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Oct 30 05:37:35.598056 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.598040    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Oct 30 05:37:35.598174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.598162    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Oct 30 05:37:35.604856 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.604835    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-d8qd8\""
Oct 30 05:37:35.604940 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.604924    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Oct 30 05:37:35.605049 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.605035    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Oct 30 05:37:35.713229 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.713190    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkntt\" (UniqueName: \"kubernetes.io/projected/1e0ae525-cf70-4563-a02d-f526915b1d48-kube-api-access-mkntt\") pod \"lws-controller-manager-5d64f7d9-9vp8v\" (UID: \"1e0ae525-cf70-4563-a02d-f526915b1d48\") " pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.713394 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.713238    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1e0ae525-cf70-4563-a02d-f526915b1d48-manager-config\") pod \"lws-controller-manager-5d64f7d9-9vp8v\" (UID: \"1e0ae525-cf70-4563-a02d-f526915b1d48\") " pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.713394 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.713262    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e0ae525-cf70-4563-a02d-f526915b1d48-cert\") pod \"lws-controller-manager-5d64f7d9-9vp8v\" (UID: \"1e0ae525-cf70-4563-a02d-f526915b1d48\") " pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.713394 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.713380    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e0ae525-cf70-4563-a02d-f526915b1d48-metrics-cert\") pod \"lws-controller-manager-5d64f7d9-9vp8v\" (UID: \"1e0ae525-cf70-4563-a02d-f526915b1d48\") " pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.814164 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.814098    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkntt\" (UniqueName: \"kubernetes.io/projected/1e0ae525-cf70-4563-a02d-f526915b1d48-kube-api-access-mkntt\") pod \"lws-controller-manager-5d64f7d9-9vp8v\" (UID: \"1e0ae525-cf70-4563-a02d-f526915b1d48\") " pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.814164 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.814138    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1e0ae525-cf70-4563-a02d-f526915b1d48-manager-config\") pod \"lws-controller-manager-5d64f7d9-9vp8v\" (UID: \"1e0ae525-cf70-4563-a02d-f526915b1d48\") " pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.814164 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.814156    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e0ae525-cf70-4563-a02d-f526915b1d48-cert\") pod \"lws-controller-manager-5d64f7d9-9vp8v\" (UID: \"1e0ae525-cf70-4563-a02d-f526915b1d48\") " pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.814394 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.814184    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e0ae525-cf70-4563-a02d-f526915b1d48-metrics-cert\") pod \"lws-controller-manager-5d64f7d9-9vp8v\" (UID: \"1e0ae525-cf70-4563-a02d-f526915b1d48\") " pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.814782 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.814761    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1e0ae525-cf70-4563-a02d-f526915b1d48-manager-config\") pod \"lws-controller-manager-5d64f7d9-9vp8v\" (UID: \"1e0ae525-cf70-4563-a02d-f526915b1d48\") " pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.816763 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.816742    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e0ae525-cf70-4563-a02d-f526915b1d48-cert\") pod \"lws-controller-manager-5d64f7d9-9vp8v\" (UID: \"1e0ae525-cf70-4563-a02d-f526915b1d48\") " pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.816838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.816791    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e0ae525-cf70-4563-a02d-f526915b1d48-metrics-cert\") pod \"lws-controller-manager-5d64f7d9-9vp8v\" (UID: \"1e0ae525-cf70-4563-a02d-f526915b1d48\") " pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.840008 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.839988    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkntt\" (UniqueName: \"kubernetes.io/projected/1e0ae525-cf70-4563-a02d-f526915b1d48-kube-api-access-mkntt\") pod \"lws-controller-manager-5d64f7d9-9vp8v\" (UID: \"1e0ae525-cf70-4563-a02d-f526915b1d48\") " pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:35.844855 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:35.844829    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:36.145947 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:36.145925    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"]
Oct 30 05:37:36.148527 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:37:36.148498    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0ae525_cf70_4563_a02d_f526915b1d48.slice/crio-5073ba0d2fb9739cde909124d97253aa9a616307f949da99bec0ecf8e4b3c830 WatchSource:0}: Error finding container 5073ba0d2fb9739cde909124d97253aa9a616307f949da99bec0ecf8e4b3c830: Status 404 returned error can't find the container with id 5073ba0d2fb9739cde909124d97253aa9a616307f949da99bec0ecf8e4b3c830
Oct 30 05:37:36.190028 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:36.189996    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v" event={"ID":"1e0ae525-cf70-4563-a02d-f526915b1d48","Type":"ContainerStarted","Data":"5073ba0d2fb9739cde909124d97253aa9a616307f949da99bec0ecf8e4b3c830"}
Oct 30 05:37:37.274652 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:37.274612    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:37.442940 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:37.442907    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:37.857519 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:37.857472    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:38.037255 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:38.037220    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:39.609262 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:39.609227    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:39.775278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:39.775242    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:41.120261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:41.120229    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:41.214939 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:41.214904    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v" event={"ID":"1e0ae525-cf70-4563-a02d-f526915b1d48","Type":"ContainerStarted","Data":"b8e3900dfca2ad4b28a40b0ad937010748f2759741e3da85c152241263dbb388"}
Oct 30 05:37:41.215123 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:41.215044    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:41.268170 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:41.268117    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v" podStartSLOduration=1.962831352 podStartE2EDuration="6.268105573s" totalImagesPullingTime="4.305274221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:37:41.267775366 +0000 UTC m=+365.013060698" watchObservedRunningTime="2026-10-30 05:37:41.268105573 +0000 UTC m=+365.013390905"
Oct 30 05:37:41.409890 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:41.409858    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:41.735939 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:41.735854    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:42.120328 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:42.120249    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:42.676186 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:42.676151    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:42.871669 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:42.871625    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:43.809194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:43.809160    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:43.973391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:43.973354    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:44.115439 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.115361    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"]
Oct 30 05:37:44.119019 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.118995    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.138853 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.138834    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 05:37:44.143704 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.143666    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"]
Oct 30 05:37:44.153266 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.153241    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-r62z4\""
Oct 30 05:37:44.242573 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.242539    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:44.287411 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.287376    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.287575 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.287422    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.287575 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.287536    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.287575 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.287568    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.287756 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.287643    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvkqt\" (UniqueName: \"kubernetes.io/projected/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-kube-api-access-cvkqt\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.287756 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.287669    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.287756 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.287738    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.287860 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.287781    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.287860 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.287799    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.388321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.388226    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.388321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.388275    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.388321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.388305    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvkqt\" (UniqueName: \"kubernetes.io/projected/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-kube-api-access-cvkqt\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.388571 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.388329    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.388571 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.388351    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.388571 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.388376    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.388571 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.388497    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.388797 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.388621    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.388797 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.388654    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.388797 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.388776    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.388957 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.388933    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.389002 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.388956    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.389037 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.389002    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.389317 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.389297    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.390866 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.390842    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.391309 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.391287    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.414054 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.414023    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.414158 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.414062    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvkqt\" (UniqueName: \"kubernetes.io/projected/e1bf555f-5bd6-4b63-8c64-04d79c058cb4-kube-api-access-cvkqt\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2\" (UID: \"e1bf555f-5bd6-4b63-8c64-04d79c058cb4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.416373 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.416339    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:44.433327 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.433307    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:44.755494 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.755472    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"]
Oct 30 05:37:44.757361 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:37:44.757332    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1bf555f_5bd6_4b63_8c64_04d79c058cb4.slice/crio-32b36f197f452651f9d9961896f272bd5312f416fe7d930c54ab7e570564af3d WatchSource:0}: Error finding container 32b36f197f452651f9d9961896f272bd5312f416fe7d930c54ab7e570564af3d: Status 404 returned error can't find the container with id 32b36f197f452651f9d9961896f272bd5312f416fe7d930c54ab7e570564af3d
Oct 30 05:37:44.762487 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:44.762465    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:45.234101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:45.234065    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2" event={"ID":"e1bf555f-5bd6-4b63-8c64-04d79c058cb4","Type":"ContainerStarted","Data":"32b36f197f452651f9d9961896f272bd5312f416fe7d930c54ab7e570564af3d"}
Oct 30 05:37:47.170061 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:47.170031    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:37:47.170331 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:47.170098    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:37:47.170331 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:47.170125    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:37:47.245018 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:47.244985    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2" event={"ID":"e1bf555f-5bd6-4b63-8c64-04d79c058cb4","Type":"ContainerStarted","Data":"ccb0555f63781fe517304e9e7d4d07015ca55033c02ace078474a4ed51f61e51"}
Oct 30 05:37:47.310698 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:47.310648    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2" podStartSLOduration=0.900914804 podStartE2EDuration="3.310634249s" totalImagesPullingTime="2.409719445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:37:47.31002392 +0000 UTC m=+371.055309251" watchObservedRunningTime="2026-10-30 05:37:47.310634249 +0000 UTC m=+371.055919575"
Oct 30 05:37:47.433636 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:47.433535    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:48.438937 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:48.438902    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:49.253208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:49.253179    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:49.254153 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:49.254128    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2"
Oct 30 05:37:52.220648 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:52.220620    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5d64f7d9-9vp8v"
Oct 30 05:37:55.250987 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:55.250946    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:37:55.434785 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:37:55.434752    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:02.550970 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:38:02.550936    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:38:20.558199 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:20.558126    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:26.219100 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:26.219068    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:32.550549 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:38:32.550513    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:38:33.800009 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:33.799977    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:33.964937 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:33.964906    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:41.809135 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:41.809101    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:41.980193 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:41.980161    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:42.038522 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.038485    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-mz8np"]
Oct 30 05:38:42.046434 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.046404    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np"
Oct 30 05:38:42.090467 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.090398    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-8wxhc\""
Oct 30 05:38:42.091995 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.091964    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18075ec4-0c86-4b95-9cf7-20c74ed041ff-cert\") pod \"kserve-controller-manager-776cc66d47-mz8np\" (UID: \"18075ec4-0c86-4b95-9cf7-20c74ed041ff\") " pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np"
Oct 30 05:38:42.092261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.092225    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw2fd\" (UniqueName: \"kubernetes.io/projected/18075ec4-0c86-4b95-9cf7-20c74ed041ff-kube-api-access-gw2fd\") pod \"kserve-controller-manager-776cc66d47-mz8np\" (UID: \"18075ec4-0c86-4b95-9cf7-20c74ed041ff\") " pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np"
Oct 30 05:38:42.104077 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.104056    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Oct 30 05:38:42.105557 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.105534    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-mz8np"]
Oct 30 05:38:42.106733 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.106715    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"]
Oct 30 05:38:42.110273 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.110258    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"
Oct 30 05:38:42.135162 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.135138    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-t86w2\""
Oct 30 05:38:42.135283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.135227    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Oct 30 05:38:42.139722 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.139699    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"]
Oct 30 05:38:42.192657 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.192626    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw2fd\" (UniqueName: \"kubernetes.io/projected/18075ec4-0c86-4b95-9cf7-20c74ed041ff-kube-api-access-gw2fd\") pod \"kserve-controller-manager-776cc66d47-mz8np\" (UID: \"18075ec4-0c86-4b95-9cf7-20c74ed041ff\") " pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np"
Oct 30 05:38:42.192818 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.192677    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-sb5xm\" (UID: \"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"
Oct 30 05:38:42.192818 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.192721    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18075ec4-0c86-4b95-9cf7-20c74ed041ff-cert\") pod \"kserve-controller-manager-776cc66d47-mz8np\" (UID: \"18075ec4-0c86-4b95-9cf7-20c74ed041ff\") " pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np"
Oct 30 05:38:42.192818 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.192753    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5n9p\" (UniqueName: \"kubernetes.io/projected/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-kube-api-access-p5n9p\") pod \"llmisvc-controller-manager-869d5dcc5d-sb5xm\" (UID: \"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"
Oct 30 05:38:42.195502 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.195456    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18075ec4-0c86-4b95-9cf7-20c74ed041ff-cert\") pod \"kserve-controller-manager-776cc66d47-mz8np\" (UID: \"18075ec4-0c86-4b95-9cf7-20c74ed041ff\") " pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np"
Oct 30 05:38:42.209078 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.209046    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-ltdvw"]
Oct 30 05:38:42.212857 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.212839    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw"
Oct 30 05:38:42.222450 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.222424    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw2fd\" (UniqueName: \"kubernetes.io/projected/18075ec4-0c86-4b95-9cf7-20c74ed041ff-kube-api-access-gw2fd\") pod \"kserve-controller-manager-776cc66d47-mz8np\" (UID: \"18075ec4-0c86-4b95-9cf7-20c74ed041ff\") " pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np"
Oct 30 05:38:42.225943 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.225914    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:42.235905 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.235882    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Oct 30 05:38:42.236085 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.236067    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-lwmmj\""
Oct 30 05:38:42.238319 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.238296    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-ltdvw"]
Oct 30 05:38:42.239664 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.239643    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-nbt6q"]
Oct 30 05:38:42.244109 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.244085    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:42.264168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.264132    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-nbt6q"]
Oct 30 05:38:42.294051 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.294023    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5n9p\" (UniqueName: \"kubernetes.io/projected/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-kube-api-access-p5n9p\") pod \"llmisvc-controller-manager-869d5dcc5d-sb5xm\" (UID: \"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"
Oct 30 05:38:42.294212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.294079    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtcwm\" (UniqueName: \"kubernetes.io/projected/e7a2529b-780f-44d4-9958-3791774aa7a5-kube-api-access-xtcwm\") pod \"model-serving-api-75bd89fdfc-ltdvw\" (UID: \"e7a2529b-780f-44d4-9958-3791774aa7a5\") " pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw"
Oct 30 05:38:42.294212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.294163    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7a2529b-780f-44d4-9958-3791774aa7a5-tls-certs\") pod \"model-serving-api-75bd89fdfc-ltdvw\" (UID: \"e7a2529b-780f-44d4-9958-3791774aa7a5\") " pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw"
Oct 30 05:38:42.294291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.294218    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-sb5xm\" (UID: \"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"
Oct 30 05:38:42.296736 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.296715    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-sb5xm\" (UID: \"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"
Oct 30 05:38:42.319470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.319441    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5n9p\" (UniqueName: \"kubernetes.io/projected/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-kube-api-access-p5n9p\") pod \"llmisvc-controller-manager-869d5dcc5d-sb5xm\" (UID: \"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"
Oct 30 05:38:42.335217 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.335193    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 30 05:38:42.335392 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.335374    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 30 05:38:42.335695 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.335680    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mxlhw\""
Oct 30 05:38:42.359174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.359114    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np"
Oct 30 05:38:42.394741 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.394706    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/730f3ec9-a3a7-4d99-8064-ee403ca67a48-cert\") pod \"odh-model-controller-f87fcf56c-nbt6q\" (UID: \"730f3ec9-a3a7-4d99-8064-ee403ca67a48\") " pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:42.394918 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.394768    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtcwm\" (UniqueName: \"kubernetes.io/projected/e7a2529b-780f-44d4-9958-3791774aa7a5-kube-api-access-xtcwm\") pod \"model-serving-api-75bd89fdfc-ltdvw\" (UID: \"e7a2529b-780f-44d4-9958-3791774aa7a5\") " pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw"
Oct 30 05:38:42.394918 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.394843    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7a2529b-780f-44d4-9958-3791774aa7a5-tls-certs\") pod \"model-serving-api-75bd89fdfc-ltdvw\" (UID: \"e7a2529b-780f-44d4-9958-3791774aa7a5\") " pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw"
Oct 30 05:38:42.394918 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.394871    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/730f3ec9-a3a7-4d99-8064-ee403ca67a48-metrics-certs\") pod \"odh-model-controller-f87fcf56c-nbt6q\" (UID: \"730f3ec9-a3a7-4d99-8064-ee403ca67a48\") " pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:42.395094 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.394950    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jstp\" (UniqueName: \"kubernetes.io/projected/730f3ec9-a3a7-4d99-8064-ee403ca67a48-kube-api-access-2jstp\") pod \"odh-model-controller-f87fcf56c-nbt6q\" (UID: \"730f3ec9-a3a7-4d99-8064-ee403ca67a48\") " pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:42.397716 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.397682    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7a2529b-780f-44d4-9958-3791774aa7a5-tls-certs\") pod \"model-serving-api-75bd89fdfc-ltdvw\" (UID: \"e7a2529b-780f-44d4-9958-3791774aa7a5\") " pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw"
Oct 30 05:38:42.399254 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.399226    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:42.422576 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.421868    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtcwm\" (UniqueName: \"kubernetes.io/projected/e7a2529b-780f-44d4-9958-3791774aa7a5-kube-api-access-xtcwm\") pod \"model-serving-api-75bd89fdfc-ltdvw\" (UID: \"e7a2529b-780f-44d4-9958-3791774aa7a5\") " pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw"
Oct 30 05:38:42.422576 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.422065    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"
Oct 30 05:38:42.495413 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.495380    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/730f3ec9-a3a7-4d99-8064-ee403ca67a48-cert\") pod \"odh-model-controller-f87fcf56c-nbt6q\" (UID: \"730f3ec9-a3a7-4d99-8064-ee403ca67a48\") " pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:42.495556 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.495477    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/730f3ec9-a3a7-4d99-8064-ee403ca67a48-metrics-certs\") pod \"odh-model-controller-f87fcf56c-nbt6q\" (UID: \"730f3ec9-a3a7-4d99-8064-ee403ca67a48\") " pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:42.495632 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.495560    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jstp\" (UniqueName: \"kubernetes.io/projected/730f3ec9-a3a7-4d99-8064-ee403ca67a48-kube-api-access-2jstp\") pod \"odh-model-controller-f87fcf56c-nbt6q\" (UID: \"730f3ec9-a3a7-4d99-8064-ee403ca67a48\") " pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:42.498792 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.498771    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/730f3ec9-a3a7-4d99-8064-ee403ca67a48-metrics-certs\") pod \"odh-model-controller-f87fcf56c-nbt6q\" (UID: \"730f3ec9-a3a7-4d99-8064-ee403ca67a48\") " pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:42.498876 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.498787    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/730f3ec9-a3a7-4d99-8064-ee403ca67a48-cert\") pod \"odh-model-controller-f87fcf56c-nbt6q\" (UID: \"730f3ec9-a3a7-4d99-8064-ee403ca67a48\") " pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:42.521524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.521493    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jstp\" (UniqueName: \"kubernetes.io/projected/730f3ec9-a3a7-4d99-8064-ee403ca67a48-kube-api-access-2jstp\") pod \"odh-model-controller-f87fcf56c-nbt6q\" (UID: \"730f3ec9-a3a7-4d99-8064-ee403ca67a48\") " pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:42.526363 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.526345    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw"
Oct 30 05:38:42.558545 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.558511    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:42.616312 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.616128    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:42.663290 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.663266    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-mz8np"]
Oct 30 05:38:42.665225 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:38:42.665201    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18075ec4_0c86_4b95_9cf7_20c74ed041ff.slice/crio-3a46c13a366afe25bd9375da555c87564c985c04d1a8c2b72d87e49f963f1405 WatchSource:0}: Error finding container 3a46c13a366afe25bd9375da555c87564c985c04d1a8c2b72d87e49f963f1405: Status 404 returned error can't find the container with id 3a46c13a366afe25bd9375da555c87564c985c04d1a8c2b72d87e49f963f1405
Oct 30 05:38:42.728786 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.728763    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"]
Oct 30 05:38:42.730850 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:38:42.730823    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9aa41fc2_7190_4f5c_a958_9ccf84fb98d0.slice/crio-2501bb5848a3649925dcbb98454827739869e3036305003460be87ce5ae4b66f WatchSource:0}: Error finding container 2501bb5848a3649925dcbb98454827739869e3036305003460be87ce5ae4b66f: Status 404 returned error can't find the container with id 2501bb5848a3649925dcbb98454827739869e3036305003460be87ce5ae4b66f
Oct 30 05:38:42.832661 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.832631    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-ltdvw"]
Oct 30 05:38:42.834173 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:38:42.834149    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a2529b_780f_44d4_9958_3791774aa7a5.slice/crio-a75ba400b7e98c3aad808b58ae7d7a8cfb175c5ebca5e82b1d571ad5cbcadbfe WatchSource:0}: Error finding container a75ba400b7e98c3aad808b58ae7d7a8cfb175c5ebca5e82b1d571ad5cbcadbfe: Status 404 returned error can't find the container with id a75ba400b7e98c3aad808b58ae7d7a8cfb175c5ebca5e82b1d571ad5cbcadbfe
Oct 30 05:38:42.871605 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:42.871570    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-nbt6q"]
Oct 30 05:38:42.873738 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:38:42.873712    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod730f3ec9_a3a7_4d99_8064_ee403ca67a48.slice/crio-8229debe78017dfad86087c930b474cd07e417e2c5312848b25b034d77ce7f49 WatchSource:0}: Error finding container 8229debe78017dfad86087c930b474cd07e417e2c5312848b25b034d77ce7f49: Status 404 returned error can't find the container with id 8229debe78017dfad86087c930b474cd07e417e2c5312848b25b034d77ce7f49
Oct 30 05:38:43.490181 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:43.490094    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q" event={"ID":"730f3ec9-a3a7-4d99-8064-ee403ca67a48","Type":"ContainerStarted","Data":"8229debe78017dfad86087c930b474cd07e417e2c5312848b25b034d77ce7f49"}
Oct 30 05:38:43.492081 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:43.492014    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np" event={"ID":"18075ec4-0c86-4b95-9cf7-20c74ed041ff","Type":"ContainerStarted","Data":"3a46c13a366afe25bd9375da555c87564c985c04d1a8c2b72d87e49f963f1405"}
Oct 30 05:38:43.505155 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:43.505120    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw" event={"ID":"e7a2529b-780f-44d4-9958-3791774aa7a5","Type":"ContainerStarted","Data":"a75ba400b7e98c3aad808b58ae7d7a8cfb175c5ebca5e82b1d571ad5cbcadbfe"}
Oct 30 05:38:43.508942 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:43.508911    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm" event={"ID":"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0","Type":"ContainerStarted","Data":"2501bb5848a3649925dcbb98454827739869e3036305003460be87ce5ae4b66f"}
Oct 30 05:38:45.453996 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:45.453957    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:45.617135 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:45.617099    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:45.775008 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:45.774853    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:45.947983 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:45.947949    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:38:46.532873 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:46.531948    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw" event={"ID":"e7a2529b-780f-44d4-9958-3791774aa7a5","Type":"ContainerStarted","Data":"43e1fbb9f2a8a3c3eea41c3231981e9b7a14885f815b9910266fcadb2de0323e"}
Oct 30 05:38:46.532873 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:46.532815    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw"
Oct 30 05:38:46.578846 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:46.578779    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw" podStartSLOduration=1.499911333 podStartE2EDuration="4.578760684s" totalImagesPullingTime="3.078849351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:38:46.578583926 +0000 UTC m=+430.323869259" watchObservedRunningTime="2026-10-30 05:38:46.578760684 +0000 UTC m=+430.324046019"
Oct 30 05:38:47.539241 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:47.539203    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np" event={"ID":"18075ec4-0c86-4b95-9cf7-20c74ed041ff","Type":"ContainerStarted","Data":"709d0dc776c296bff918d3848a6d8fb76ee8be0f021d0d14aff1d3b9c1d0d93f"}
Oct 30 05:38:47.539785 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:47.539765    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np"
Oct 30 05:38:47.541405 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:47.541170    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm" event={"ID":"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0","Type":"ContainerStarted","Data":"89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b"}
Oct 30 05:38:47.541405 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:47.541267    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"
Oct 30 05:38:47.542643 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:47.542622    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q" event={"ID":"730f3ec9-a3a7-4d99-8064-ee403ca67a48","Type":"ContainerStarted","Data":"695325d4cef840c4e167bdce65c2ed7007b5a91db575a8610548f69ef824331a"}
Oct 30 05:38:47.542950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:47.542930    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:47.598439 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:47.598387    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np" podStartSLOduration=1.4484744410000001 podStartE2EDuration="5.598373397s" totalImagesPullingTime="4.149898956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:38:47.596880727 +0000 UTC m=+431.342166072" watchObservedRunningTime="2026-10-30 05:38:47.598373397 +0000 UTC m=+431.343658729"
Oct 30 05:38:47.644321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:47.644268    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q" podStartSLOduration=1.611824613 podStartE2EDuration="5.644256115s" totalImagesPullingTime="4.032431502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:38:47.64326222 +0000 UTC m=+431.388547551" watchObservedRunningTime="2026-10-30 05:38:47.644256115 +0000 UTC m=+431.389541483"
Oct 30 05:38:58.549926 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:58.549894    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-f87fcf56c-nbt6q"
Oct 30 05:38:58.552039 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:58.552019    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-ltdvw"
Oct 30 05:38:58.599085 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:38:58.599039    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm" podStartSLOduration=12.472345782 podStartE2EDuration="16.599025156s" totalImagesPullingTime="4.126679374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:38:47.690774534 +0000 UTC m=+431.436059863" watchObservedRunningTime="2026-10-30 05:38:58.599025156 +0000 UTC m=+442.344310513"
Oct 30 05:39:02.550133 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:39:02.550099    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:39:18.549676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:39:18.549641    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"
Oct 30 05:39:18.553394 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:39:18.553373    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-mz8np"
Oct 30 05:39:26.350876 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:39:26.350835    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:39:30.861841 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:39:30.861809    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:39:31.045563 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:39:31.045532    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:39:32.551218 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:39:32.551187    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:39:52.271957 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:39:52.271872    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:39:52.435497 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:39:52.435465    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:39:57.819747 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:39:57.819713    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:01.826808 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:01.826773    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:02.550498 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:40:02.550463    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:40:03.317104 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.317075    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq"]
Oct 30 05:40:03.319536 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.319521    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq"
Oct 30 05:40:03.343464 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.343407    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-df9lf\""
Oct 30 05:40:03.343666 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.343650    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 05:40:03.346738 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.346717    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 05:40:03.359329 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.359302    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq"]
Oct 30 05:40:03.410947 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.410913    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrvg7\" (UniqueName: \"kubernetes.io/projected/3692525c-1db2-4a7a-ba04-c26bdcf52282-kube-api-access-mrvg7\") pod \"dns-operator-controller-manager-65bfd79598-6k4sq\" (UID: \"3692525c-1db2-4a7a-ba04-c26bdcf52282\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq"
Oct 30 05:40:03.443533 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.443500    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 05:40:03.511480 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.511451    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrvg7\" (UniqueName: \"kubernetes.io/projected/3692525c-1db2-4a7a-ba04-c26bdcf52282-kube-api-access-mrvg7\") pod \"dns-operator-controller-manager-65bfd79598-6k4sq\" (UID: \"3692525c-1db2-4a7a-ba04-c26bdcf52282\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq"
Oct 30 05:40:03.543077 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.543037    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrvg7\" (UniqueName: \"kubernetes.io/projected/3692525c-1db2-4a7a-ba04-c26bdcf52282-kube-api-access-mrvg7\") pod \"dns-operator-controller-manager-65bfd79598-6k4sq\" (UID: \"3692525c-1db2-4a7a-ba04-c26bdcf52282\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq"
Oct 30 05:40:03.630745 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.630660    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq"
Oct 30 05:40:03.644888 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.644861    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:03.945838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:03.945814    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq"]
Oct 30 05:40:03.948113 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:40:03.948088    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3692525c_1db2_4a7a_ba04_c26bdcf52282.slice/crio-1eee7f37f6499aa1508c69d80acc02659e03c28c3a002f63f0332ff2f04a2e04 WatchSource:0}: Error finding container 1eee7f37f6499aa1508c69d80acc02659e03c28c3a002f63f0332ff2f04a2e04: Status 404 returned error can't find the container with id 1eee7f37f6499aa1508c69d80acc02659e03c28c3a002f63f0332ff2f04a2e04
Oct 30 05:40:04.022860 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:04.022825    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:04.281408 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:04.281332    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:04.514123 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:04.514079    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:04.893687 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:04.893654    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq" event={"ID":"3692525c-1db2-4a7a-ba04-c26bdcf52282","Type":"ContainerStarted","Data":"1eee7f37f6499aa1508c69d80acc02659e03c28c3a002f63f0332ff2f04a2e04"}
Oct 30 05:40:05.662788 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:05.662756    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:05.958268 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:05.958191    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-4lplv"]
Oct 30 05:40:05.961353 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:05.961334    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d6dd76dc-4lplv"
Oct 30 05:40:06.002081 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:06.002056    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-9r7qk\""
Oct 30 05:40:06.003511 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:06.003490    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-4lplv"]
Oct 30 05:40:06.037800 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:06.037775    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v76kj\" (UniqueName: \"kubernetes.io/projected/d25dce9a-0499-413d-99c7-0766aaf63fa1-kube-api-access-v76kj\") pod \"authorino-operator-68d6dd76dc-4lplv\" (UID: \"d25dce9a-0499-413d-99c7-0766aaf63fa1\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-4lplv"
Oct 30 05:40:06.138765 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:06.138724    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v76kj\" (UniqueName: \"kubernetes.io/projected/d25dce9a-0499-413d-99c7-0766aaf63fa1-kube-api-access-v76kj\") pod \"authorino-operator-68d6dd76dc-4lplv\" (UID: \"d25dce9a-0499-413d-99c7-0766aaf63fa1\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-4lplv"
Oct 30 05:40:06.164995 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:06.164971    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v76kj\" (UniqueName: \"kubernetes.io/projected/d25dce9a-0499-413d-99c7-0766aaf63fa1-kube-api-access-v76kj\") pod \"authorino-operator-68d6dd76dc-4lplv\" (UID: \"d25dce9a-0499-413d-99c7-0766aaf63fa1\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-4lplv"
Oct 30 05:40:06.274562 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:06.274484    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d6dd76dc-4lplv"
Oct 30 05:40:06.574157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:06.574139    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-4lplv"]
Oct 30 05:40:06.576337 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:40:06.576312    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd25dce9a_0499_413d_99c7_0766aaf63fa1.slice/crio-78601c2f40a05a99294a5011e22ca5a8ac2899cf4eaf37ec45466b0fd6acd527 WatchSource:0}: Error finding container 78601c2f40a05a99294a5011e22ca5a8ac2899cf4eaf37ec45466b0fd6acd527: Status 404 returned error can't find the container with id 78601c2f40a05a99294a5011e22ca5a8ac2899cf4eaf37ec45466b0fd6acd527
Oct 30 05:40:06.630411 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:06.630371    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:06.852491 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:06.852388    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:06.904602 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:06.904538    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d6dd76dc-4lplv" event={"ID":"d25dce9a-0499-413d-99c7-0766aaf63fa1","Type":"ContainerStarted","Data":"78601c2f40a05a99294a5011e22ca5a8ac2899cf4eaf37ec45466b0fd6acd527"}
Oct 30 05:40:07.034877 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:07.034842    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:07.638467 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:07.638431    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:07.958402 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:07.958362    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:08.157415 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:08.157370    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:09.937116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:09.937078    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d6dd76dc-4lplv" event={"ID":"d25dce9a-0499-413d-99c7-0766aaf63fa1","Type":"ContainerStarted","Data":"3c509c1c56985d3b669f02c73f9dc7f7140e4a29c3761630fd5f4ec0831d3742"}
Oct 30 05:40:09.937712 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:09.937146    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d6dd76dc-4lplv"
Oct 30 05:40:09.938844 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:09.938820    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq" event={"ID":"3692525c-1db2-4a7a-ba04-c26bdcf52282","Type":"ContainerStarted","Data":"edd54641630cda253bb596c239b86243a18808b4e4191b94f35e5d529462a0e2"}
Oct 30 05:40:09.938953 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:09.938939    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq"
Oct 30 05:40:09.992898 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:09.992846    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d6dd76dc-4lplv" podStartSLOduration=2.285499072 podStartE2EDuration="4.99283117s" totalImagesPullingTime="2.707332098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:40:09.991208959 +0000 UTC m=+513.736494293" watchObservedRunningTime="2026-10-30 05:40:09.99283117 +0000 UTC m=+513.738116502"
Oct 30 05:40:10.037618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:10.037538    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq" podStartSLOduration=1.5604998270000001 podStartE2EDuration="7.037524951s" totalImagesPullingTime="5.477025124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:40:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:40:10.036861873 +0000 UTC m=+513.782147208" watchObservedRunningTime="2026-10-30 05:40:10.037524951 +0000 UTC m=+513.782810283"
Oct 30 05:40:12.639267 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:12.639229    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:12.819023 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:12.818986    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:14.642584 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:14.642548    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:14.802815 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:14.802782    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:18.278046 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:18.278013    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:18.616033 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:18.615955    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:20.442114 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:20.442077    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:20.605931 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:20.605901    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:20.947576 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:20.947550    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-6k4sq"
Oct 30 05:40:20.947808 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:20.947640    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d6dd76dc-4lplv"
Oct 30 05:40:25.263580 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:25.263548    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:25.427679 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:25.427644    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:26.467402 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:26.467367    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:32.550397 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:40:32.550359    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:40:36.043419 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:36.043388    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:36.237080 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:36.237030    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:38.248634 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:38.248575    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:38.429768 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:38.429733    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:38.786813 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:38.786775    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:39.009566 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:39.009533    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:39.717299 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:39.717270    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:44.339262 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:44.339228    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:44.574770 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:44.574740    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:48.895166 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:48.895137    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:49.073455 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:49.073422    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:59.503631 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:59.503576    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:40:59.663256 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:40:59.663222    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:00.534840 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:00.534813    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:00.719817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:00.719781    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:02.550857 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:41:02.550753    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:41:13.151442 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:13.151408    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:13.323999 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:13.323961    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:14.039054 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:14.039019    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:14.202122 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:14.202089    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:14.406645 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:14.406618    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-xggh2"]
Oct 30 05:41:14.410312 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:14.410293    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-xggh2"
Oct 30 05:41:14.437930 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:14.437905    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-xggh2"]
Oct 30 05:41:14.449803 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:14.449777    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sffvg\""
Oct 30 05:41:14.541405 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:14.541370    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tszc7\" (UniqueName: \"kubernetes.io/projected/5c85210f-27a9-4a6f-97dd-bfd2d50b43e7-kube-api-access-tszc7\") pod \"authorino-56c6c9ddd5-xggh2\" (UID: \"5c85210f-27a9-4a6f-97dd-bfd2d50b43e7\") " pod="kuadrant-system/authorino-56c6c9ddd5-xggh2"
Oct 30 05:41:14.642494 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:14.642462    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tszc7\" (UniqueName: \"kubernetes.io/projected/5c85210f-27a9-4a6f-97dd-bfd2d50b43e7-kube-api-access-tszc7\") pod \"authorino-56c6c9ddd5-xggh2\" (UID: \"5c85210f-27a9-4a6f-97dd-bfd2d50b43e7\") " pod="kuadrant-system/authorino-56c6c9ddd5-xggh2"
Oct 30 05:41:14.668261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:14.668195    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tszc7\" (UniqueName: \"kubernetes.io/projected/5c85210f-27a9-4a6f-97dd-bfd2d50b43e7-kube-api-access-tszc7\") pod \"authorino-56c6c9ddd5-xggh2\" (UID: \"5c85210f-27a9-4a6f-97dd-bfd2d50b43e7\") " pod="kuadrant-system/authorino-56c6c9ddd5-xggh2"
Oct 30 05:41:14.734755 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:14.734720    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-xggh2"
Oct 30 05:41:15.034457 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:15.034433    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-xggh2"]
Oct 30 05:41:15.036911 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:41:15.036884    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c85210f_27a9_4a6f_97dd_bfd2d50b43e7.slice/crio-f92917786e0ec264d028a6405f7d762ce39728151ff14e079810dd5a5365ff1e WatchSource:0}: Error finding container f92917786e0ec264d028a6405f7d762ce39728151ff14e079810dd5a5365ff1e: Status 404 returned error can't find the container with id f92917786e0ec264d028a6405f7d762ce39728151ff14e079810dd5a5365ff1e
Oct 30 05:41:15.241245 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:15.241214    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-xggh2" event={"ID":"5c85210f-27a9-4a6f-97dd-bfd2d50b43e7","Type":"ContainerStarted","Data":"f92917786e0ec264d028a6405f7d762ce39728151ff14e079810dd5a5365ff1e"}
Oct 30 05:41:17.253680 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:17.253645    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-xggh2" event={"ID":"5c85210f-27a9-4a6f-97dd-bfd2d50b43e7","Type":"ContainerStarted","Data":"7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382"}
Oct 30 05:41:17.297001 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:17.296952    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-xggh2" podStartSLOduration=1.96361699 podStartE2EDuration="3.296939262s" totalImagesPullingTime="1.333322272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:41:17.296495076 +0000 UTC m=+581.041780408" watchObservedRunningTime="2026-10-30 05:41:17.296939262 +0000 UTC m=+581.042224594"
Oct 30 05:41:26.613335 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:26.613301    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:32.389985 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:32.389952    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:32.553093 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:41:32.553056    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:41:32.560456 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:32.560433    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:33.278664 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:33.278637    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:36.447485 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:36.447454    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:41:36.448691 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:36.448669    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:41:45.655603 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:45.655569    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:45.815637 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:45.815582    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:46.118937 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.118856    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-ngdz5"]
Oct 30 05:41:46.124670 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.124648    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-ngdz5"
Oct 30 05:41:46.141973 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.141943    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-ngdz5"]
Oct 30 05:41:46.154638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.154613    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 05:41:46.225873 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.225844    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldnxz\" (UniqueName: \"kubernetes.io/projected/1005467e-56aa-4cb7-917e-a52cc0b9c636-kube-api-access-ldnxz\") pod \"authorino-6889cbd8c4-ngdz5\" (UID: \"1005467e-56aa-4cb7-917e-a52cc0b9c636\") " pod="kuadrant-system/authorino-6889cbd8c4-ngdz5"
Oct 30 05:41:46.226041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.225886    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1005467e-56aa-4cb7-917e-a52cc0b9c636-tls-cert\") pod \"authorino-6889cbd8c4-ngdz5\" (UID: \"1005467e-56aa-4cb7-917e-a52cc0b9c636\") " pod="kuadrant-system/authorino-6889cbd8c4-ngdz5"
Oct 30 05:41:46.278219 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.278188    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-ngdz5"]
Oct 30 05:41:46.278405 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:41:46.278388    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-ldnxz tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6889cbd8c4-ngdz5" podUID="1005467e-56aa-4cb7-917e-a52cc0b9c636"
Oct 30 05:41:46.308739 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.308696    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-594988f7d5-gsbmp"]
Oct 30 05:41:46.312870 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.312855    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594988f7d5-gsbmp"
Oct 30 05:41:46.326456 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.326432    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldnxz\" (UniqueName: \"kubernetes.io/projected/1005467e-56aa-4cb7-917e-a52cc0b9c636-kube-api-access-ldnxz\") pod \"authorino-6889cbd8c4-ngdz5\" (UID: \"1005467e-56aa-4cb7-917e-a52cc0b9c636\") " pod="kuadrant-system/authorino-6889cbd8c4-ngdz5"
Oct 30 05:41:46.326621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.326471    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1005467e-56aa-4cb7-917e-a52cc0b9c636-tls-cert\") pod \"authorino-6889cbd8c4-ngdz5\" (UID: \"1005467e-56aa-4cb7-917e-a52cc0b9c636\") " pod="kuadrant-system/authorino-6889cbd8c4-ngdz5"
Oct 30 05:41:46.329258 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.329230    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1005467e-56aa-4cb7-917e-a52cc0b9c636-tls-cert\") pod \"authorino-6889cbd8c4-ngdz5\" (UID: \"1005467e-56aa-4cb7-917e-a52cc0b9c636\") " pod="kuadrant-system/authorino-6889cbd8c4-ngdz5"
Oct 30 05:41:46.331796 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.331773    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-594988f7d5-gsbmp"]
Oct 30 05:41:46.357486 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.357456    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldnxz\" (UniqueName: \"kubernetes.io/projected/1005467e-56aa-4cb7-917e-a52cc0b9c636-kube-api-access-ldnxz\") pod \"authorino-6889cbd8c4-ngdz5\" (UID: \"1005467e-56aa-4cb7-917e-a52cc0b9c636\") " pod="kuadrant-system/authorino-6889cbd8c4-ngdz5"
Oct 30 05:41:46.389211 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.389126    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-ngdz5"
Oct 30 05:41:46.397229 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.397203    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-ngdz5"
Oct 30 05:41:46.427085 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.427054    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/40766c09-63b8-41f1-951a-e28607a419fd-tls-cert\") pod \"authorino-594988f7d5-gsbmp\" (UID: \"40766c09-63b8-41f1-951a-e28607a419fd\") " pod="kuadrant-system/authorino-594988f7d5-gsbmp"
Oct 30 05:41:46.427241 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.427110    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgwss\" (UniqueName: \"kubernetes.io/projected/40766c09-63b8-41f1-951a-e28607a419fd-kube-api-access-wgwss\") pod \"authorino-594988f7d5-gsbmp\" (UID: \"40766c09-63b8-41f1-951a-e28607a419fd\") " pod="kuadrant-system/authorino-594988f7d5-gsbmp"
Oct 30 05:41:46.527860 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.527826    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1005467e-56aa-4cb7-917e-a52cc0b9c636-tls-cert\" (UniqueName: \"kubernetes.io/secret/1005467e-56aa-4cb7-917e-a52cc0b9c636-tls-cert\") pod \"1005467e-56aa-4cb7-917e-a52cc0b9c636\" (UID: \"1005467e-56aa-4cb7-917e-a52cc0b9c636\") "
Oct 30 05:41:46.528045 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.527875    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1005467e-56aa-4cb7-917e-a52cc0b9c636-kube-api-access-ldnxz\" (UniqueName: \"kubernetes.io/projected/1005467e-56aa-4cb7-917e-a52cc0b9c636-kube-api-access-ldnxz\") pod \"1005467e-56aa-4cb7-917e-a52cc0b9c636\" (UID: \"1005467e-56aa-4cb7-917e-a52cc0b9c636\") "
Oct 30 05:41:46.528117 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.528040    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/40766c09-63b8-41f1-951a-e28607a419fd-tls-cert\") pod \"authorino-594988f7d5-gsbmp\" (UID: \"40766c09-63b8-41f1-951a-e28607a419fd\") " pod="kuadrant-system/authorino-594988f7d5-gsbmp"
Oct 30 05:41:46.528172 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.528110    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgwss\" (UniqueName: \"kubernetes.io/projected/40766c09-63b8-41f1-951a-e28607a419fd-kube-api-access-wgwss\") pod \"authorino-594988f7d5-gsbmp\" (UID: \"40766c09-63b8-41f1-951a-e28607a419fd\") " pod="kuadrant-system/authorino-594988f7d5-gsbmp"
Oct 30 05:41:46.530146 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.530116    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1005467e-56aa-4cb7-917e-a52cc0b9c636-tls-cert" pod "1005467e-56aa-4cb7-917e-a52cc0b9c636" (UID: "1005467e-56aa-4cb7-917e-a52cc0b9c636"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:41:46.530278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.530240    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1005467e-56aa-4cb7-917e-a52cc0b9c636-kube-api-access-ldnxz" pod "1005467e-56aa-4cb7-917e-a52cc0b9c636" (UID: "1005467e-56aa-4cb7-917e-a52cc0b9c636"). InnerVolumeSpecName "kube-api-access-ldnxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:41:46.530570 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.530547    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/40766c09-63b8-41f1-951a-e28607a419fd-tls-cert\") pod \"authorino-594988f7d5-gsbmp\" (UID: \"40766c09-63b8-41f1-951a-e28607a419fd\") " pod="kuadrant-system/authorino-594988f7d5-gsbmp"
Oct 30 05:41:46.553432 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.553408    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgwss\" (UniqueName: \"kubernetes.io/projected/40766c09-63b8-41f1-951a-e28607a419fd-kube-api-access-wgwss\") pod \"authorino-594988f7d5-gsbmp\" (UID: \"40766c09-63b8-41f1-951a-e28607a419fd\") " pod="kuadrant-system/authorino-594988f7d5-gsbmp"
Oct 30 05:41:46.624872 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.624847    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594988f7d5-gsbmp"
Oct 30 05:41:46.628853 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.628637    5066 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1005467e-56aa-4cb7-917e-a52cc0b9c636-tls-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:41:46.628853 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.628657    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldnxz\" (UniqueName: \"kubernetes.io/projected/1005467e-56aa-4cb7-917e-a52cc0b9c636-kube-api-access-ldnxz\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:41:46.927490 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:46.927466    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-594988f7d5-gsbmp"]
Oct 30 05:41:46.929787 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:41:46.929760    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40766c09_63b8_41f1_951a_e28607a419fd.slice/crio-a4a2128f7e78dc755f53f53091ff41d6b2a87fbe0f768fac6d36aaa2e1761982 WatchSource:0}: Error finding container a4a2128f7e78dc755f53f53091ff41d6b2a87fbe0f768fac6d36aaa2e1761982: Status 404 returned error can't find the container with id a4a2128f7e78dc755f53f53091ff41d6b2a87fbe0f768fac6d36aaa2e1761982
Oct 30 05:41:47.396549 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:47.396503    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-594988f7d5-gsbmp" event={"ID":"40766c09-63b8-41f1-951a-e28607a419fd","Type":"ContainerStarted","Data":"12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6"}
Oct 30 05:41:47.396549 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:47.396553    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-594988f7d5-gsbmp" event={"ID":"40766c09-63b8-41f1-951a-e28607a419fd","Type":"ContainerStarted","Data":"a4a2128f7e78dc755f53f53091ff41d6b2a87fbe0f768fac6d36aaa2e1761982"}
Oct 30 05:41:47.396810 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:47.396553    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-ngdz5"
Oct 30 05:41:47.441102 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:47.441046    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-594988f7d5-gsbmp" podStartSLOduration=1.068281265 podStartE2EDuration="1.44103269s" totalImagesPullingTime="372.751425ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:41:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:41:47.439971925 +0000 UTC m=+611.185257257" watchObservedRunningTime="2026-10-30 05:41:47.44103269 +0000 UTC m=+611.186318022"
Oct 30 05:41:47.466923 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:47.466894    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-xggh2"]
Oct 30 05:41:47.467079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:47.467059    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-xggh2" podUID="5c85210f-27a9-4a6f-97dd-bfd2d50b43e7" containerName="authorino" containerID="cri-o://7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382" gracePeriod=30
Oct 30 05:41:47.516851 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:47.516817    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-ngdz5"]
Oct 30 05:41:47.523635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:47.523607    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-ngdz5"]
Oct 30 05:41:47.717682 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:47.717648    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-xggh2"
Oct 30 05:41:47.837948 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:47.837877    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c85210f-27a9-4a6f-97dd-bfd2d50b43e7-kube-api-access-tszc7\" (UniqueName: \"kubernetes.io/projected/5c85210f-27a9-4a6f-97dd-bfd2d50b43e7-kube-api-access-tszc7\") pod \"5c85210f-27a9-4a6f-97dd-bfd2d50b43e7\" (UID: \"5c85210f-27a9-4a6f-97dd-bfd2d50b43e7\") "
Oct 30 05:41:47.840181 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:47.840158    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c85210f-27a9-4a6f-97dd-bfd2d50b43e7-kube-api-access-tszc7" pod "5c85210f-27a9-4a6f-97dd-bfd2d50b43e7" (UID: "5c85210f-27a9-4a6f-97dd-bfd2d50b43e7"). InnerVolumeSpecName "kube-api-access-tszc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:41:47.939509 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:47.939479    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tszc7\" (UniqueName: \"kubernetes.io/projected/5c85210f-27a9-4a6f-97dd-bfd2d50b43e7-kube-api-access-tszc7\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:41:48.401294 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:48.401258    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1005467e-56aa-4cb7-917e-a52cc0b9c636" path="/var/lib/kubelet/pods/1005467e-56aa-4cb7-917e-a52cc0b9c636/volumes"
Oct 30 05:41:48.402386 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:48.402366    5066 generic.go:356] "Generic (PLEG): container finished" podID="5c85210f-27a9-4a6f-97dd-bfd2d50b43e7" containerID="7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382" exitCode=0
Oct 30 05:41:48.402477 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:48.402419    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-xggh2"
Oct 30 05:41:48.402477 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:48.402454    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-xggh2" event={"ID":"5c85210f-27a9-4a6f-97dd-bfd2d50b43e7","Type":"ContainerDied","Data":"7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382"}
Oct 30 05:41:48.402545 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:48.402498    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-xggh2" event={"ID":"5c85210f-27a9-4a6f-97dd-bfd2d50b43e7","Type":"ContainerDied","Data":"f92917786e0ec264d028a6405f7d762ce39728151ff14e079810dd5a5365ff1e"}
Oct 30 05:41:48.402545 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:48.402515    5066 scope.go:122] "RemoveContainer" containerID="7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382"
Oct 30 05:41:48.411339 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:48.411318    5066 scope.go:122] "RemoveContainer" containerID="7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382"
Oct 30 05:41:48.411656 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:41:48.411632    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382\": container with ID starting with 7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382 not found: ID does not exist" containerID="7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382"
Oct 30 05:41:48.411740 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:48.411678    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382"} err="failed to get container status \"7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382\": rpc error: code = NotFound desc = could not find container \"7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382\": container with ID starting with 7db45500b37dcc626152c160c3b167a2d598ffe9db07dfc9ffca590263d94382 not found: ID does not exist"
Oct 30 05:41:48.459896 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:48.459867    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-xggh2"]
Oct 30 05:41:48.465683 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:48.465658    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-xggh2"]
Oct 30 05:41:50.400707 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:50.400675    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c85210f-27a9-4a6f-97dd-bfd2d50b43e7" path="/var/lib/kubelet/pods/5c85210f-27a9-4a6f-97dd-bfd2d50b43e7/volumes"
Oct 30 05:41:54.853465 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:54.853425    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:54.875265 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:54.875236    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-577f596567-7mzv2"]
Oct 30 05:41:54.875744 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:54.875727    5066 cpu_manager.go:414] "containerMap: removing container" podUID="5c85210f-27a9-4a6f-97dd-bfd2d50b43e7" containerName="authorino"
Oct 30 05:41:54.875852 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:54.875746    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c85210f-27a9-4a6f-97dd-bfd2d50b43e7" containerName="authorino"
Oct 30 05:41:54.875852 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:54.875836    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c85210f-27a9-4a6f-97dd-bfd2d50b43e7" containerName="authorino"
Oct 30 05:41:54.880335 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:54.880313    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:41:54.911287 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:54.911254    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-7mzv2"]
Oct 30 05:41:54.923061 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:54.923039    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dcmpd\""
Oct 30 05:41:54.923061 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:54.923046    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 05:41:54.923282 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:54.923077    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 05:41:54.991228 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:54.991195    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 05:41:55.000516 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.000495    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-metrics-certs\") pod \"maas-controller-577f596567-7mzv2\" (UID: \"1b354dba-9f87-49e3-940a-4287078d417a\") " pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:41:55.000620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.000556    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-webhook-cert\") pod \"maas-controller-577f596567-7mzv2\" (UID: \"1b354dba-9f87-49e3-940a-4287078d417a\") " pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:41:55.000688 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.000671    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mx28\" (UniqueName: \"kubernetes.io/projected/1b354dba-9f87-49e3-940a-4287078d417a-kube-api-access-8mx28\") pod \"maas-controller-577f596567-7mzv2\" (UID: \"1b354dba-9f87-49e3-940a-4287078d417a\") " pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:41:55.043965 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.043933    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:55.101797 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.101770    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-webhook-cert\") pod \"maas-controller-577f596567-7mzv2\" (UID: \"1b354dba-9f87-49e3-940a-4287078d417a\") " pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:41:55.101948 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.101809    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mx28\" (UniqueName: \"kubernetes.io/projected/1b354dba-9f87-49e3-940a-4287078d417a-kube-api-access-8mx28\") pod \"maas-controller-577f596567-7mzv2\" (UID: \"1b354dba-9f87-49e3-940a-4287078d417a\") " pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:41:55.101948 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.101875    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-metrics-certs\") pod \"maas-controller-577f596567-7mzv2\" (UID: \"1b354dba-9f87-49e3-940a-4287078d417a\") " pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:41:55.104660 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.104579    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-webhook-cert\") pod \"maas-controller-577f596567-7mzv2\" (UID: \"1b354dba-9f87-49e3-940a-4287078d417a\") " pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:41:55.104660 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.104579    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-metrics-certs\") pod \"maas-controller-577f596567-7mzv2\" (UID: \"1b354dba-9f87-49e3-940a-4287078d417a\") " pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:41:55.127129 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.127103    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mx28\" (UniqueName: \"kubernetes.io/projected/1b354dba-9f87-49e3-940a-4287078d417a-kube-api-access-8mx28\") pod \"maas-controller-577f596567-7mzv2\" (UID: \"1b354dba-9f87-49e3-940a-4287078d417a\") " pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:41:55.198738 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.198708    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:41:55.210339 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.210308    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:55.243487 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.243455    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5974d5d96-flgp2"]
Oct 30 05:41:55.247212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.247188    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:41:55.274624 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.272650    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5974d5d96-flgp2"]
Oct 30 05:41:55.404123 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.404101    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-metrics-certs\") pod \"maas-controller-5974d5d96-flgp2\" (UID: \"1b95079a-128b-45eb-be31-d240aeccea74\") " pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:41:55.404272 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.404160    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksxt2\" (UniqueName: \"kubernetes.io/projected/1b95079a-128b-45eb-be31-d240aeccea74-kube-api-access-ksxt2\") pod \"maas-controller-5974d5d96-flgp2\" (UID: \"1b95079a-128b-45eb-be31-d240aeccea74\") " pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:41:55.404312 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.404289    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-webhook-cert\") pod \"maas-controller-5974d5d96-flgp2\" (UID: \"1b95079a-128b-45eb-be31-d240aeccea74\") " pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:41:55.504912 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.504872    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-webhook-cert\") pod \"maas-controller-5974d5d96-flgp2\" (UID: \"1b95079a-128b-45eb-be31-d240aeccea74\") " pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:41:55.504973 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.504941    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-metrics-certs\") pod \"maas-controller-5974d5d96-flgp2\" (UID: \"1b95079a-128b-45eb-be31-d240aeccea74\") " pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:41:55.505310 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.505030    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksxt2\" (UniqueName: \"kubernetes.io/projected/1b95079a-128b-45eb-be31-d240aeccea74-kube-api-access-ksxt2\") pod \"maas-controller-5974d5d96-flgp2\" (UID: \"1b95079a-128b-45eb-be31-d240aeccea74\") " pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:41:55.505428 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.505392    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-7mzv2"]
Oct 30 05:41:55.507636 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.507610    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-webhook-cert\") pod \"maas-controller-5974d5d96-flgp2\" (UID: \"1b95079a-128b-45eb-be31-d240aeccea74\") " pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:41:55.507747 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:41:55.507677    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b354dba_9f87_49e3_940a_4287078d417a.slice/crio-5a9b32ab658ca6d49d4abd5ac94e507ad070085f6e8eb9729668e4e5858933e1 WatchSource:0}: Error finding container 5a9b32ab658ca6d49d4abd5ac94e507ad070085f6e8eb9729668e4e5858933e1: Status 404 returned error can't find the container with id 5a9b32ab658ca6d49d4abd5ac94e507ad070085f6e8eb9729668e4e5858933e1
Oct 30 05:41:55.507817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.507790    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-metrics-certs\") pod \"maas-controller-5974d5d96-flgp2\" (UID: \"1b95079a-128b-45eb-be31-d240aeccea74\") " pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:41:55.530840 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.530812    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksxt2\" (UniqueName: \"kubernetes.io/projected/1b95079a-128b-45eb-be31-d240aeccea74-kube-api-access-ksxt2\") pod \"maas-controller-5974d5d96-flgp2\" (UID: \"1b95079a-128b-45eb-be31-d240aeccea74\") " pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:41:55.575272 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.575243    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:41:55.881397 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:55.881370    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5974d5d96-flgp2"]
Oct 30 05:41:55.881957 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:41:55.881930    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b95079a_128b_45eb_be31_d240aeccea74.slice/crio-b2be97de7ca56ad17d49db13180bab68c1905e80f12f7c4b11d5d0de0023845a WatchSource:0}: Error finding container b2be97de7ca56ad17d49db13180bab68c1905e80f12f7c4b11d5d0de0023845a: Status 404 returned error can't find the container with id b2be97de7ca56ad17d49db13180bab68c1905e80f12f7c4b11d5d0de0023845a
Oct 30 05:41:56.443913 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:56.443874    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5974d5d96-flgp2" event={"ID":"1b95079a-128b-45eb-be31-d240aeccea74","Type":"ContainerStarted","Data":"b2be97de7ca56ad17d49db13180bab68c1905e80f12f7c4b11d5d0de0023845a"}
Oct 30 05:41:56.446667 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:56.446639    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-7mzv2" event={"ID":"1b354dba-9f87-49e3-940a-4287078d417a","Type":"ContainerStarted","Data":"5a9b32ab658ca6d49d4abd5ac94e507ad070085f6e8eb9729668e4e5858933e1"}
Oct 30 05:41:58.462235 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:58.462199    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-7mzv2" event={"ID":"1b354dba-9f87-49e3-940a-4287078d417a","Type":"ContainerStarted","Data":"5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e"}
Oct 30 05:41:58.462663 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:58.462324    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:41:58.530300 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:58.530251    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-577f596567-7mzv2" podStartSLOduration=1.942411479 podStartE2EDuration="4.530238503s" totalImagesPullingTime="2.587827024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:41:58.529559064 +0000 UTC m=+622.274844397" watchObservedRunningTime="2026-10-30 05:41:58.530238503 +0000 UTC m=+622.275523835"
Oct 30 05:41:58.558920 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:58.558892    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:58.770670 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:58.770632    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:41:59.471024 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:59.470985    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5974d5d96-flgp2" event={"ID":"1b95079a-128b-45eb-be31-d240aeccea74","Type":"ContainerStarted","Data":"131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654"}
Oct 30 05:41:59.471415 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:59.471077    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:41:59.520927 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:41:59.520881    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5974d5d96-flgp2" podStartSLOduration=2.021962524 podStartE2EDuration="4.520864943s" totalImagesPullingTime="2.498902419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:41:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:41:59.51861716 +0000 UTC m=+623.263902493" watchObservedRunningTime="2026-10-30 05:41:59.520864943 +0000 UTC m=+623.266150292"
Oct 30 05:42:01.904267 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:01.904233    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"]
Oct 30 05:42:01.907858 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:01.907841    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:01.948276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:01.948246    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 05:42:01.948468 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:01.948304    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 05:42:01.948468 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:01.948317    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-xbp7n\""
Oct 30 05:42:01.948468 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:01.948365    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 05:42:01.950023 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:01.949999    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"]
Oct 30 05:42:02.068124 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.068096    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/1c5632fa-9bfe-424f-a58f-0cb02928fc63-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-nlrqh\" (UID: \"1c5632fa-9bfe-424f-a58f-0cb02928fc63\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.068277 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.068146    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwggs\" (UniqueName: \"kubernetes.io/projected/1c5632fa-9bfe-424f-a58f-0cb02928fc63-kube-api-access-kwggs\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-nlrqh\" (UID: \"1c5632fa-9bfe-424f-a58f-0cb02928fc63\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.068277 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.068171    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5632fa-9bfe-424f-a58f-0cb02928fc63-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-nlrqh\" (UID: \"1c5632fa-9bfe-424f-a58f-0cb02928fc63\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.068398 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.068303    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c5632fa-9bfe-424f-a58f-0cb02928fc63-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-nlrqh\" (UID: \"1c5632fa-9bfe-424f-a58f-0cb02928fc63\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.071786 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.071760    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:42:02.168838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.168752    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5632fa-9bfe-424f-a58f-0cb02928fc63-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-nlrqh\" (UID: \"1c5632fa-9bfe-424f-a58f-0cb02928fc63\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.168838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.168816    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c5632fa-9bfe-424f-a58f-0cb02928fc63-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-nlrqh\" (UID: \"1c5632fa-9bfe-424f-a58f-0cb02928fc63\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.169030 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.168876    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/1c5632fa-9bfe-424f-a58f-0cb02928fc63-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-nlrqh\" (UID: \"1c5632fa-9bfe-424f-a58f-0cb02928fc63\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.169030 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.168906    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwggs\" (UniqueName: \"kubernetes.io/projected/1c5632fa-9bfe-424f-a58f-0cb02928fc63-kube-api-access-kwggs\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-nlrqh\" (UID: \"1c5632fa-9bfe-424f-a58f-0cb02928fc63\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.169544 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.169523    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c5632fa-9bfe-424f-a58f-0cb02928fc63-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-nlrqh\" (UID: \"1c5632fa-9bfe-424f-a58f-0cb02928fc63\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.169837 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.169636    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/1c5632fa-9bfe-424f-a58f-0cb02928fc63-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-nlrqh\" (UID: \"1c5632fa-9bfe-424f-a58f-0cb02928fc63\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.171407 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.171386    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c5632fa-9bfe-424f-a58f-0cb02928fc63-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-nlrqh\" (UID: \"1c5632fa-9bfe-424f-a58f-0cb02928fc63\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.194161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.194130    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwggs\" (UniqueName: \"kubernetes.io/projected/1c5632fa-9bfe-424f-a58f-0cb02928fc63-kube-api-access-kwggs\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-nlrqh\" (UID: \"1c5632fa-9bfe-424f-a58f-0cb02928fc63\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.221773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.221731    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:02.271295 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.271258    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:42:02.526791 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:02.526696    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"]
Oct 30 05:42:02.529765 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:42:02.529732    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c5632fa_9bfe_424f_a58f_0cb02928fc63.slice/crio-236446d6a03159bf871edd9a790de3176f5f64fd3a44891a6d68e9b0e45a14ad WatchSource:0}: Error finding container 236446d6a03159bf871edd9a790de3176f5f64fd3a44891a6d68e9b0e45a14ad: Status 404 returned error can't find the container with id 236446d6a03159bf871edd9a790de3176f5f64fd3a44891a6d68e9b0e45a14ad
Oct 30 05:42:02.552713 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:42:02.552683    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:42:03.492008 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:03.491977    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh" event={"ID":"1c5632fa-9bfe-424f-a58f-0cb02928fc63","Type":"ContainerStarted","Data":"236446d6a03159bf871edd9a790de3176f5f64fd3a44891a6d68e9b0e45a14ad"}
Oct 30 05:42:09.473250 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:09.473214    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:42:10.478967 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:10.478937    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 05:42:10.560098 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:10.560069    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-7mzv2"]
Oct 30 05:42:10.560308 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:10.560261    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-577f596567-7mzv2" podUID="1b354dba-9f87-49e3-940a-4287078d417a" containerName="manager" containerID="cri-o://5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e" gracePeriod=30
Oct 30 05:42:10.852008 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:10.851965    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:42:10.953905 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:10.953872    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-metrics-certs\") pod \"1b354dba-9f87-49e3-940a-4287078d417a\" (UID: \"1b354dba-9f87-49e3-940a-4287078d417a\") "
Oct 30 05:42:10.954101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:10.953922    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b354dba-9f87-49e3-940a-4287078d417a-kube-api-access-8mx28\" (UniqueName: \"kubernetes.io/projected/1b354dba-9f87-49e3-940a-4287078d417a-kube-api-access-8mx28\") pod \"1b354dba-9f87-49e3-940a-4287078d417a\" (UID: \"1b354dba-9f87-49e3-940a-4287078d417a\") "
Oct 30 05:42:10.954101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:10.953981    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-webhook-cert\") pod \"1b354dba-9f87-49e3-940a-4287078d417a\" (UID: \"1b354dba-9f87-49e3-940a-4287078d417a\") "
Oct 30 05:42:10.959306 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:10.959256    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-webhook-cert" pod "1b354dba-9f87-49e3-940a-4287078d417a" (UID: "1b354dba-9f87-49e3-940a-4287078d417a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:42:10.959691 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:10.959662    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b354dba-9f87-49e3-940a-4287078d417a-kube-api-access-8mx28" pod "1b354dba-9f87-49e3-940a-4287078d417a" (UID: "1b354dba-9f87-49e3-940a-4287078d417a"). InnerVolumeSpecName "kube-api-access-8mx28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:42:10.961179 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:10.961143    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-metrics-certs" pod "1b354dba-9f87-49e3-940a-4287078d417a" (UID: "1b354dba-9f87-49e3-940a-4287078d417a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:42:11.058141 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:11.058059    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:42:11.058141 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:11.058099    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mx28\" (UniqueName: \"kubernetes.io/projected/1b354dba-9f87-49e3-940a-4287078d417a-kube-api-access-8mx28\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:42:11.058141 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:11.058115    5066 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b354dba-9f87-49e3-940a-4287078d417a-webhook-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:42:11.540062 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:11.540020    5066 generic.go:356] "Generic (PLEG): container finished" podID="1b354dba-9f87-49e3-940a-4287078d417a" containerID="5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e" exitCode=0
Oct 30 05:42:11.540573 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:11.540090    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-7mzv2"
Oct 30 05:42:11.540573 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:11.540107    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-7mzv2" event={"ID":"1b354dba-9f87-49e3-940a-4287078d417a","Type":"ContainerDied","Data":"5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e"}
Oct 30 05:42:11.540573 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:11.540176    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-7mzv2" event={"ID":"1b354dba-9f87-49e3-940a-4287078d417a","Type":"ContainerDied","Data":"5a9b32ab658ca6d49d4abd5ac94e507ad070085f6e8eb9729668e4e5858933e1"}
Oct 30 05:42:11.540573 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:11.540207    5066 scope.go:122] "RemoveContainer" containerID="5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e"
Oct 30 05:42:11.551264 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:11.551245    5066 scope.go:122] "RemoveContainer" containerID="5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e"
Oct 30 05:42:11.551614 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:42:11.551575    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e\": container with ID starting with 5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e not found: ID does not exist" containerID="5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e"
Oct 30 05:42:11.551718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:11.551627    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e"} err="failed to get container status \"5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e\": rpc error: code = NotFound desc = could not find container \"5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e\": container with ID starting with 5b0cf0f0e705f73e073818ebd335221e938fe53e0bde6f983733f99ead1fe24e not found: ID does not exist"
Oct 30 05:42:11.598329 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:11.598296    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-7mzv2"]
Oct 30 05:42:11.612251 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:11.612190    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-577f596567-7mzv2"]
Oct 30 05:42:12.402317 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:12.402282    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b354dba-9f87-49e3-940a-4287078d417a" path="/var/lib/kubelet/pods/1b354dba-9f87-49e3-940a-4287078d417a/volumes"
Oct 30 05:42:13.554535 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:13.554493    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh" event={"ID":"1c5632fa-9bfe-424f-a58f-0cb02928fc63","Type":"ContainerStarted","Data":"17ae17d9bacc4a75bd725074a328a9746e9de88ec050a4a41a5dd290113a13db"}
Oct 30 05:42:13.554940 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:13.554629    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:13.603545 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:13.603499    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh" podStartSLOduration=2.480898379 podStartE2EDuration="12.603482664s" totalImagesPullingTime="10.122584285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:42:13.603200464 +0000 UTC m=+637.348485796" watchObservedRunningTime="2026-10-30 05:42:13.603482664 +0000 UTC m=+637.348767998"
Oct 30 05:42:24.564566 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:24.564526    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-nlrqh"
Oct 30 05:42:26.719455 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:26.719423    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:42:31.870813 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:31.870782    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:42:32.170030 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:32.169999    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:42:32.551633 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:42:32.551526    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:42:33.173811 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:33.173765    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:42:33.378070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:33.378029    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:42:33.547077 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:33.546991    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:42:33.835539 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:33.835468    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:42:44.492496 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:44.492458    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"]
Oct 30 05:42:44.493131 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:44.493055    5066 cpu_manager.go:414] "containerMap: removing container" podUID="1b354dba-9f87-49e3-940a-4287078d417a" containerName="manager"
Oct 30 05:42:44.493131 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:44.493085    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b354dba-9f87-49e3-940a-4287078d417a" containerName="manager"
Oct 30 05:42:44.493257 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:44.493232    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b354dba-9f87-49e3-940a-4287078d417a" containerName="manager"
Oct 30 05:42:44.844258 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:44.844172    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"]
Oct 30 05:42:44.844435 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:44.844284    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"
Oct 30 05:42:44.975404 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:44.975365    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea7416de-1581-4482-b9c8-3615dc434510-cert\") pod \"llmisvc-controller-manager-6466c554d8-hbjqb\" (UID: \"ea7416de-1581-4482-b9c8-3615dc434510\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"
Oct 30 05:42:44.975573 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:44.975415    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f2kv\" (UniqueName: \"kubernetes.io/projected/ea7416de-1581-4482-b9c8-3615dc434510-kube-api-access-4f2kv\") pod \"llmisvc-controller-manager-6466c554d8-hbjqb\" (UID: \"ea7416de-1581-4482-b9c8-3615dc434510\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"
Oct 30 05:42:45.076131 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:45.076091    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea7416de-1581-4482-b9c8-3615dc434510-cert\") pod \"llmisvc-controller-manager-6466c554d8-hbjqb\" (UID: \"ea7416de-1581-4482-b9c8-3615dc434510\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"
Oct 30 05:42:45.076348 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:45.076141    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f2kv\" (UniqueName: \"kubernetes.io/projected/ea7416de-1581-4482-b9c8-3615dc434510-kube-api-access-4f2kv\") pod \"llmisvc-controller-manager-6466c554d8-hbjqb\" (UID: \"ea7416de-1581-4482-b9c8-3615dc434510\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"
Oct 30 05:42:45.078738 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:45.078704    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea7416de-1581-4482-b9c8-3615dc434510-cert\") pod \"llmisvc-controller-manager-6466c554d8-hbjqb\" (UID: \"ea7416de-1581-4482-b9c8-3615dc434510\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"
Oct 30 05:42:45.102393 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:45.102321    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f2kv\" (UniqueName: \"kubernetes.io/projected/ea7416de-1581-4482-b9c8-3615dc434510-kube-api-access-4f2kv\") pod \"llmisvc-controller-manager-6466c554d8-hbjqb\" (UID: \"ea7416de-1581-4482-b9c8-3615dc434510\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"
Oct 30 05:42:45.180978 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:45.180945    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"
Oct 30 05:42:45.544996 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:45.544972    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"]
Oct 30 05:42:45.546765 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:42:45.546739    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea7416de_1581_4482_b9c8_3615dc434510.slice/crio-5cfb34009acd0eedf29f3af93529f3f1efb9b97976d7183843a880c7a4f63831 WatchSource:0}: Error finding container 5cfb34009acd0eedf29f3af93529f3f1efb9b97976d7183843a880c7a4f63831: Status 404 returned error can't find the container with id 5cfb34009acd0eedf29f3af93529f3f1efb9b97976d7183843a880c7a4f63831
Oct 30 05:42:45.548243 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:45.548222    5066 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 05:42:45.704402 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:45.704363    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb" event={"ID":"ea7416de-1581-4482-b9c8-3615dc434510","Type":"ContainerStarted","Data":"5cfb34009acd0eedf29f3af93529f3f1efb9b97976d7183843a880c7a4f63831"}
Oct 30 05:42:46.710533 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:46.710458    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb" event={"ID":"ea7416de-1581-4482-b9c8-3615dc434510","Type":"ContainerStarted","Data":"2b13600cec30a3e1d52b5e5b95547e0879e6ed306c035d912607d98d329040f1"}
Oct 30 05:42:46.710952 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:46.710547    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"
Oct 30 05:42:46.757229 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:42:46.757178    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb" podStartSLOduration=2.313051915 podStartE2EDuration="2.757163398s" totalImagesPullingTime="444.111483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:42:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:42:46.755813293 +0000 UTC m=+670.501098629" watchObservedRunningTime="2026-10-30 05:42:46.757163398 +0000 UTC m=+670.502448781"
Oct 30 05:43:02.551313 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:43:02.551275    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:43:17.720077 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:17.720048    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hbjqb"
Oct 30 05:43:17.796634 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:17.796603    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"]
Oct 30 05:43:17.860796 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:17.860735    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm" podUID="9aa41fc2-7190-4f5c-a958-9ccf84fb98d0" containerName="manager" containerID="cri-o://89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b" gracePeriod=30
Oct 30 05:43:18.112458 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.112434    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"
Oct 30 05:43:18.284878 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.284798    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-cert\" (UniqueName: \"kubernetes.io/secret/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-cert\") pod \"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0\" (UID: \"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0\") "
Oct 30 05:43:18.284878 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.284830    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-kube-api-access-p5n9p\" (UniqueName: \"kubernetes.io/projected/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-kube-api-access-p5n9p\") pod \"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0\" (UID: \"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0\") "
Oct 30 05:43:18.292664 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.292629    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-kube-api-access-p5n9p" pod "9aa41fc2-7190-4f5c-a958-9ccf84fb98d0" (UID: "9aa41fc2-7190-4f5c-a958-9ccf84fb98d0"). InnerVolumeSpecName "kube-api-access-p5n9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:43:18.292957 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.292925    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-cert" pod "9aa41fc2-7190-4f5c-a958-9ccf84fb98d0" (UID: "9aa41fc2-7190-4f5c-a958-9ccf84fb98d0"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:43:18.386030 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.385996    5066 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:43:18.386190 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.386047    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5n9p\" (UniqueName: \"kubernetes.io/projected/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0-kube-api-access-p5n9p\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:43:18.734908 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.734870    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:43:18.867008 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.866968    5066 generic.go:356] "Generic (PLEG): container finished" podID="9aa41fc2-7190-4f5c-a958-9ccf84fb98d0" containerID="89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b" exitCode=0
Oct 30 05:43:18.867178 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.867037    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"
Oct 30 05:43:18.867178 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.867046    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm" event={"ID":"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0","Type":"ContainerDied","Data":"89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b"}
Oct 30 05:43:18.867178 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.867106    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm" event={"ID":"9aa41fc2-7190-4f5c-a958-9ccf84fb98d0","Type":"ContainerDied","Data":"2501bb5848a3649925dcbb98454827739869e3036305003460be87ce5ae4b66f"}
Oct 30 05:43:18.867178 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.867136    5066 scope.go:122] "RemoveContainer" containerID="89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b"
Oct 30 05:43:18.876611 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.876570    5066 scope.go:122] "RemoveContainer" containerID="89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b"
Oct 30 05:43:18.876874 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:43:18.876856    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b\": container with ID starting with 89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b not found: ID does not exist" containerID="89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b"
Oct 30 05:43:18.876933 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.876883    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b"} err="failed to get container status \"89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b\": rpc error: code = NotFound desc = could not find container \"89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b\": container with ID starting with 89fd035011e14dbc9bd1ac4d1efe4e6bb3011e1a6589d5e7502804426d8c2d5b not found: ID does not exist"
Oct 30 05:43:18.926471 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.926443    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"]
Oct 30 05:43:18.931018 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.930995    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:43:18.940480 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:18.940459    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-sb5xm"]
Oct 30 05:43:20.400843 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:20.400806    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9aa41fc2-7190-4f5c-a958-9ccf84fb98d0" path="/var/lib/kubelet/pods/9aa41fc2-7190-4f5c-a958-9ccf84fb98d0/volumes"
Oct 30 05:43:26.837527 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:26.837496    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:43:32.550710 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:43:32.550668    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:43:34.613546 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:34.613506    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:43:36.695623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:36.695555    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:43:42.675092 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:42.675049    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:43:44.628936 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:44.628888    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:43:46.464388 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:46.464353    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:43:48.558669 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:48.558637    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:43:50.074819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:43:50.074790    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:44:01.370258 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.370221    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"]
Oct 30 05:44:01.370712 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.370676    5066 cpu_manager.go:414] "containerMap: removing container" podUID="9aa41fc2-7190-4f5c-a958-9ccf84fb98d0" containerName="manager"
Oct 30 05:44:01.370712 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.370688    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9aa41fc2-7190-4f5c-a958-9ccf84fb98d0" containerName="manager"
Oct 30 05:44:01.370817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.370806    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="9aa41fc2-7190-4f5c-a958-9ccf84fb98d0" containerName="manager"
Oct 30 05:44:01.374276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.374255    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.391367 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.391346    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ntk7q\""
Oct 30 05:44:01.407221 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.407191    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"]
Oct 30 05:44:01.494833 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.494806    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.495007 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.494839    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.495007 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.494940    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.495007 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.494964    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.495007 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.494983    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kbbb\" (UniqueName: \"kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-kube-api-access-7kbbb\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.495167 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.495007    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.495167 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.495096    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.495167 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.495125    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.495167 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.495142    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596081    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596113    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596134    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kbbb\" (UniqueName: \"kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-kube-api-access-7kbbb\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596157    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596185    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596227    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596252    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596298    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596326    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596772 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596556    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596772 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596645    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596772 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596761    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.596914 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.596832    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.597037 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.597019    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.598820 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.598790    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.599071 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.599054    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.622558 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.622491    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.622744 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.622723    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kbbb\" (UniqueName: \"kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-kube-api-access-7kbbb\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xfps7\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:01.687213 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:01.687178    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:02.011144 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:02.011119    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"]
Oct 30 05:44:02.013006 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:44:02.012980    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2736e286_37f0_4c52_9e22_eab3c3b52b5e.slice/crio-15b53e84d6a1da8cd44f923b57bf8d9c88c6f0d326d80a1b463aabdee8be0dee WatchSource:0}: Error finding container 15b53e84d6a1da8cd44f923b57bf8d9c88c6f0d326d80a1b463aabdee8be0dee: Status 404 returned error can't find the container with id 15b53e84d6a1da8cd44f923b57bf8d9c88c6f0d326d80a1b463aabdee8be0dee
Oct 30 05:44:02.015064 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:02.015033    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:44:02.015169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:02.015108    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:44:02.015169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:02.015150    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:44:02.084283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:02.084243    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" event={"ID":"2736e286-37f0-4c52-9e22-eab3c3b52b5e","Type":"ContainerStarted","Data":"1d92f01ed1cac08614493d8af497d915f93a845f76bee51ef4830dfc2e59ac7e"}
Oct 30 05:44:02.084437 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:02.084317    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" event={"ID":"2736e286-37f0-4c52-9e22-eab3c3b52b5e","Type":"ContainerStarted","Data":"15b53e84d6a1da8cd44f923b57bf8d9c88c6f0d326d80a1b463aabdee8be0dee"}
Oct 30 05:44:02.134581 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:02.134533    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" podStartSLOduration=1.13451986 podStartE2EDuration="1.13451986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:44:02.134117813 +0000 UTC m=+745.879403146" watchObservedRunningTime="2026-10-30 05:44:02.13451986 +0000 UTC m=+745.879805192"
Oct 30 05:44:02.575417 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:44:02.575384    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:44:02.688075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:02.688047    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:03.689644 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:03.689576    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xfps7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 05:44:03.690033 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:03.689659    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 05:44:04.687965 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:04.687923    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xfps7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 05:44:04.688165 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:04.687984    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 05:44:05.688464 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:05.688426    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xfps7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 05:44:05.689039 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:05.688486    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 05:44:06.688062 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:06.688024    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xfps7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 05:44:06.688266 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:06.688096    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 05:44:07.687546 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:07.687504    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xfps7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 05:44:07.688025 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:07.687571    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 05:44:07.759291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:07.759258    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-594988f7d5-gsbmp"]
Oct 30 05:44:07.759460 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:07.759433    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-594988f7d5-gsbmp" podUID="40766c09-63b8-41f1-951a-e28607a419fd" containerName="authorino" containerID="cri-o://12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6" gracePeriod=30
Oct 30 05:44:08.019772 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.019745    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594988f7d5-gsbmp"
Oct 30 05:44:08.117771 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.117738    5066 generic.go:356] "Generic (PLEG): container finished" podID="40766c09-63b8-41f1-951a-e28607a419fd" containerID="12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6" exitCode=0
Oct 30 05:44:08.117942 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.117781    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594988f7d5-gsbmp"
Oct 30 05:44:08.117942 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.117822    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-594988f7d5-gsbmp" event={"ID":"40766c09-63b8-41f1-951a-e28607a419fd","Type":"ContainerDied","Data":"12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6"}
Oct 30 05:44:08.117942 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.117864    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-594988f7d5-gsbmp" event={"ID":"40766c09-63b8-41f1-951a-e28607a419fd","Type":"ContainerDied","Data":"a4a2128f7e78dc755f53f53091ff41d6b2a87fbe0f768fac6d36aaa2e1761982"}
Oct 30 05:44:08.117942 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.117885    5066 scope.go:122] "RemoveContainer" containerID="12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6"
Oct 30 05:44:08.128739 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.128713    5066 scope.go:122] "RemoveContainer" containerID="12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6"
Oct 30 05:44:08.129071 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:44:08.129053    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6\": container with ID starting with 12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6 not found: ID does not exist" containerID="12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6"
Oct 30 05:44:08.129122 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.129084    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6"} err="failed to get container status \"12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6\": rpc error: code = NotFound desc = could not find container \"12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6\": container with ID starting with 12a87b78b5575280dcfbac68def195dd0365c34e92e032d84e2a1010d08c6aa6 not found: ID does not exist"
Oct 30 05:44:08.179119 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.179084    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40766c09-63b8-41f1-951a-e28607a419fd-kube-api-access-wgwss\" (UniqueName: \"kubernetes.io/projected/40766c09-63b8-41f1-951a-e28607a419fd-kube-api-access-wgwss\") pod \"40766c09-63b8-41f1-951a-e28607a419fd\" (UID: \"40766c09-63b8-41f1-951a-e28607a419fd\") "
Oct 30 05:44:08.179275 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.179123    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40766c09-63b8-41f1-951a-e28607a419fd-tls-cert\" (UniqueName: \"kubernetes.io/secret/40766c09-63b8-41f1-951a-e28607a419fd-tls-cert\") pod \"40766c09-63b8-41f1-951a-e28607a419fd\" (UID: \"40766c09-63b8-41f1-951a-e28607a419fd\") "
Oct 30 05:44:08.181406 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.181381    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40766c09-63b8-41f1-951a-e28607a419fd-kube-api-access-wgwss" pod "40766c09-63b8-41f1-951a-e28607a419fd" (UID: "40766c09-63b8-41f1-951a-e28607a419fd"). InnerVolumeSpecName "kube-api-access-wgwss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:44:08.192541 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.192519    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40766c09-63b8-41f1-951a-e28607a419fd-tls-cert" pod "40766c09-63b8-41f1-951a-e28607a419fd" (UID: "40766c09-63b8-41f1-951a-e28607a419fd"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:44:08.280134 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.280058    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgwss\" (UniqueName: \"kubernetes.io/projected/40766c09-63b8-41f1-951a-e28607a419fd-kube-api-access-wgwss\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:44:08.280134 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.280087    5066 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/40766c09-63b8-41f1-951a-e28607a419fd-tls-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:44:08.477955 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.477907    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-594988f7d5-gsbmp"]
Oct 30 05:44:08.489408 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.489380    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-594988f7d5-gsbmp"]
Oct 30 05:44:08.507249 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.507217    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:44:08.688053 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.688021    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xfps7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 05:44:08.688443 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.688081    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 05:44:08.697420 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:08.697382    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:44:10.310618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:10.310564    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:10.311321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:10.311261    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:10.312074 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:10.312057    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 05:44:10.400577 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:10.400542    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40766c09-63b8-41f1-951a-e28607a419fd" path="/var/lib/kubelet/pods/40766c09-63b8-41f1-951a-e28607a419fd/volumes"
Oct 30 05:44:13.477400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:13.477371    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:44:13.681388 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:13.681355    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:44:26.962070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:26.962034    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:44:31.457900 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.457854    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"]
Oct 30 05:44:31.458497 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.458477    5066 cpu_manager.go:414] "containerMap: removing container" podUID="40766c09-63b8-41f1-951a-e28607a419fd" containerName="authorino"
Oct 30 05:44:31.458497 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.458495    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40766c09-63b8-41f1-951a-e28607a419fd" containerName="authorino"
Oct 30 05:44:31.458671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.458579    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="40766c09-63b8-41f1-951a-e28607a419fd" containerName="authorino"
Oct 30 05:44:31.462375 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.462329    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.497450 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.497409    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"]
Oct 30 05:44:31.499021 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.498999    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"]
Oct 30 05:44:31.502399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.502379    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.519709 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.519682    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"]
Oct 30 05:44:31.591969 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.591933    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/204b23d9-e64b-475c-be79-4a3746371200-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.591969 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.591972    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.592229 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.591991    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.592229 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592052    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.592229 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592149    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/204b23d9-e64b-475c-be79-4a3746371200-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.592229 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592196    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.592407 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592246    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv9fc\" (UniqueName: \"kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-kube-api-access-nv9fc\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.592407 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592308    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.592407 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592330    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.592407 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592344    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfp5x\" (UniqueName: \"kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-kube-api-access-lfp5x\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.592407 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592361    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.592691 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592469    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.592691 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592501    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.592691 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592567    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.592691 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592639    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.592838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592725    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.592838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592770    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.592838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.592790    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.693493 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693459    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.693493 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693499    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/204b23d9-e64b-475c-be79-4a3746371200-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.693758 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693527    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.693758 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693557    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.693758 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693582    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.693758 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693652    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/204b23d9-e64b-475c-be79-4a3746371200-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.693758 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693693    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.693758 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693717    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv9fc\" (UniqueName: \"kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-kube-api-access-nv9fc\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.693758 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693743    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.694116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693775    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.694116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693801    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfp5x\" (UniqueName: \"kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-kube-api-access-lfp5x\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.694116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693825    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.694116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693856    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.694116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693881    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.694116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693909    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.694116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.693940    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.694116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.694072    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.694116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.694074    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.694578 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.694211    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.694578 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.694387    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.694578 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.694421    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/204b23d9-e64b-475c-be79-4a3746371200-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.694578 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.694538    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.694981 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.694584    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.694981 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.694686    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.694981 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.694831    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.695140 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.695070    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.695267 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.695137    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.695542 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.695519    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.696416 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.696387    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.696526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.696458    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/204b23d9-e64b-475c-be79-4a3746371200-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.696526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.696512    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.696702 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.696684    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.719697 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.719629    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.719697 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.719659    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.719697 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.719668    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv9fc\" (UniqueName: \"kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-kube-api-access-nv9fc\") pod \"maas-default-gateway-openshift-default-58847f9fc5-xjhtt\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:31.719879 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.719814    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfp5x\" (UniqueName: \"kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-kube-api-access-lfp5x\") pod \"maas-default-gateway-openshift-default-58847f9fc5-p99cc\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.776775 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.776744    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:31.816395 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:31.816360    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:32.098619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.098576    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"]
Oct 30 05:44:32.100678 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:44:32.100649    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod204b23d9_e64b_475c_be79_4a3746371200.slice/crio-c8e31ec710cf286ddc0e01307632da18043c2bef585a0bc0a683211ccb397b74 WatchSource:0}: Error finding container c8e31ec710cf286ddc0e01307632da18043c2bef585a0bc0a683211ccb397b74: Status 404 returned error can't find the container with id c8e31ec710cf286ddc0e01307632da18043c2bef585a0bc0a683211ccb397b74
Oct 30 05:44:32.102847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.102817    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:44:32.102904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.102878    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:44:32.102947 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.102918    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:44:32.145304 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.145281    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"]
Oct 30 05:44:32.147156 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:44:32.147126    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd59ca3c_8f24_4f58_bd47_01e6eae62873.slice/crio-7c519783d975324f35a2bbd7be9dbc8988e2eb59070db03d8d4502bfebb17e08 WatchSource:0}: Error finding container 7c519783d975324f35a2bbd7be9dbc8988e2eb59070db03d8d4502bfebb17e08: Status 404 returned error can't find the container with id 7c519783d975324f35a2bbd7be9dbc8988e2eb59070db03d8d4502bfebb17e08
Oct 30 05:44:32.149298 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.149269    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:44:32.149358 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.149338    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:44:32.149395 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.149368    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:44:32.256324 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.255988    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" event={"ID":"cd59ca3c-8f24-4f58-bd47-01e6eae62873","Type":"ContainerStarted","Data":"ce4307ba80c4bf2564ac9ae5f16d90d905b5085383f2092bff694220d5e6186d"}
Oct 30 05:44:32.256490 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.256415    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" event={"ID":"cd59ca3c-8f24-4f58-bd47-01e6eae62873","Type":"ContainerStarted","Data":"7c519783d975324f35a2bbd7be9dbc8988e2eb59070db03d8d4502bfebb17e08"}
Oct 30 05:44:32.263078 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.263041    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" event={"ID":"204b23d9-e64b-475c-be79-4a3746371200","Type":"ContainerStarted","Data":"eb9dfff5edd2f1dadc273e34f832919c628c0a989ac53d263f6025cc93ee78c8"}
Oct 30 05:44:32.263239 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.263104    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" event={"ID":"204b23d9-e64b-475c-be79-4a3746371200","Type":"ContainerStarted","Data":"c8e31ec710cf286ddc0e01307632da18043c2bef585a0bc0a683211ccb397b74"}
Oct 30 05:44:32.315843 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.315780    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" podStartSLOduration=1.315759602 podStartE2EDuration="1.315759602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:44:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:44:32.314557879 +0000 UTC m=+776.059843248" watchObservedRunningTime="2026-10-30 05:44:32.315759602 +0000 UTC m=+776.061044934"
Oct 30 05:44:32.369766 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.369642    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" podStartSLOduration=1.369620983 podStartE2EDuration="1.369620983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:44:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:44:32.367461848 +0000 UTC m=+776.112747181" watchObservedRunningTime="2026-10-30 05:44:32.369620983 +0000 UTC m=+776.114906317"
Oct 30 05:44:32.571665 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:44:32.571620    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:44:32.778611 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.777773    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:32.816964 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:32.816923    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:33.779538 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:33.779510    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-p99cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 05:44:33.780211 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:33.780186    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 05:44:33.818080 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:33.818056    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xjhtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 05:44:33.818283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:33.818265    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 05:44:34.779971 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:34.779943    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-p99cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 05:44:34.780431 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:34.780409    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 05:44:34.816995 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:34.816966    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xjhtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 05:44:34.817133 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:34.817011    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 05:44:35.778415 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:35.778077    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-p99cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 05:44:35.778415 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:35.778139    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 05:44:35.816981 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:35.816900    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xjhtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 05:44:35.816981 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:35.816942    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 05:44:36.777720 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:36.777627    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-p99cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 05:44:36.777720 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:36.777695    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 05:44:36.817508 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:36.817480    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xjhtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 05:44:36.817984 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:36.817958    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 05:44:37.702126 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:37.702092    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:44:37.777359 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:37.777322    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-p99cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 05:44:37.777544 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:37.777392    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 05:44:37.816837 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:37.816800    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xjhtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 05:44:37.816990 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:37.816851    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 05:44:38.252470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:38.252428    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:44:38.559205 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:38.559117    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:44:38.777627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:38.777572    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-p99cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 05:44:38.777805 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:38.777662    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 05:44:38.817233 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:38.817148    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xjhtt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 05:44:38.817233 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:38.817195    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 05:44:39.851893 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:39.851852    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:44:40.056157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:40.056112    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:44:40.573263 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:40.573235    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:40.573879 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:40.573852    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:40.574633 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:40.574613    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 05:44:40.617992 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:40.617964    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:40.618542 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:40.618522    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:40.619204 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:40.619184    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 05:44:41.389236 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:41.389203    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:44:41.630675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:44:41.630627    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:02.551028 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:45:02.550991    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:45:04.093152 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:04.093110    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:04.252966 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:04.252934    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:04.591858 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:04.591821    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:04.761374 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:04.761344    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:05.011823 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:05.011790    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:05.192735 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:05.192703    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:05.377661 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:05.377555    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:05.543549 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:05.543515    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:05.707321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:05.707263    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:05.868925 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:05.868890    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:08.572319 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:08.572267    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:08.725962 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:08.725917    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:08.961483 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:08.961454    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:09.180220 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:09.180186    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:09.356117 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:09.356043    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:09.554116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:09.554086    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:09.736501 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:09.736468    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:10.257550 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:10.257523    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:10.482237 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:10.482197    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:10.654525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:10.654490    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:11.815425 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:11.815395    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:11.983975 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:11.983929    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:12.205066 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:12.205018    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:12.570699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:12.570578    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:12.764200 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:12.764171    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:13.898659 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:13.898619    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:14.067093 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:14.067059    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:14.643455 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:14.643427    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:14.803745 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:14.803693    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:15.443059 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.443028    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:15.732066 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.731992    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"]
Oct 30 05:45:15.737287 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.737265    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:15.755833 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.755811    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"]
Oct 30 05:45:15.767976 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.767954    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-a35835-serving-cert\""
Oct 30 05:45:15.768147 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.768121    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 05:45:15.768222 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.768046    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4vtg8\""
Oct 30 05:45:15.840787 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.840747    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 05:45:15.840961 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.840836    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 05:45:15.870010 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.869970    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-maas-api-tls\") pod \"maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:15.870010 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.870019    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-metrics-certs\") pod \"maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:15.870235 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.870044    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cf69466-534b-4646-99ba-f1bbb28fb92b-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:15.870281 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.870244    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ng2c\" (UniqueName: \"kubernetes.io/projected/4cf69466-534b-4646-99ba-f1bbb28fb92b-kube-api-access-5ng2c\") pod \"maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:15.971390 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.971356    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ng2c\" (UniqueName: \"kubernetes.io/projected/4cf69466-534b-4646-99ba-f1bbb28fb92b-kube-api-access-5ng2c\") pod \"maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:15.971390 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.971395    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-maas-api-tls\") pod \"maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:15.971709 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.971419    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-metrics-certs\") pod \"maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:15.971709 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.971551    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cf69466-534b-4646-99ba-f1bbb28fb92b-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:15.972289 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.972236    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cf69466-534b-4646-99ba-f1bbb28fb92b-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:15.974289 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.974265    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-metrics-certs\") pod \"maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:15.974504 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.974457    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-maas-api-tls\") pod \"maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:15.997047 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:15.996983    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ng2c\" (UniqueName: \"kubernetes.io/projected/4cf69466-534b-4646-99ba-f1bbb28fb92b-kube-api-access-5ng2c\") pod \"maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:16.049870 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:16.049835    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:16.354073 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:16.354050    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"]
Oct 30 05:45:16.355918 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:45:16.355890    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cf69466_534b_4646_99ba_f1bbb28fb92b.slice/crio-6f0be8c20417aaa334383a788cb7824b7734f6aade67ca4d49b21ae7313a3051 WatchSource:0}: Error finding container 6f0be8c20417aaa334383a788cb7824b7734f6aade67ca4d49b21ae7313a3051: Status 404 returned error can't find the container with id 6f0be8c20417aaa334383a788cb7824b7734f6aade67ca4d49b21ae7313a3051
Oct 30 05:45:16.496303 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:16.496264    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr" event={"ID":"4cf69466-534b-4646-99ba-f1bbb28fb92b","Type":"ContainerStarted","Data":"6f0be8c20417aaa334383a788cb7824b7734f6aade67ca4d49b21ae7313a3051"}
Oct 30 05:45:16.635819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:16.635740    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:16.795359 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:16.795323    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:16.954483 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:16.954451    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:17.494186 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:17.494153    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:17.667505 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:17.667459    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:17.834892 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:17.834840    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:17.997133 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:17.997090    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:18.509274 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:18.509238    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr" event={"ID":"4cf69466-534b-4646-99ba-f1bbb28fb92b","Type":"ContainerStarted","Data":"c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2"}
Oct 30 05:45:18.509450 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:18.509357    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:18.558427 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:18.558379    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr" podStartSLOduration=2.1757793850000002 podStartE2EDuration="3.558366474s" totalImagesPullingTime="1.382587089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:45:18.558122785 +0000 UTC m=+822.303408118" watchObservedRunningTime="2026-10-30 05:45:18.558366474 +0000 UTC m=+822.303651815"
Oct 30 05:45:19.439196 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.439163    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"]
Oct 30 05:45:19.442986 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.442966    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.475236 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.475206    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-4a68a1-serving-cert\""
Oct 30 05:45:19.477155 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.477123    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"]
Oct 30 05:45:19.593862 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.593831    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:19.613900 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.613867    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-maas-api-tls\") pod \"maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.614062 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.613973    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6chkj\" (UniqueName: \"kubernetes.io/projected/b1193647-74ea-437e-b1ae-9040d7440d48-kube-api-access-6chkj\") pod \"maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.614062 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.614055    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1193647-74ea-437e-b1ae-9040d7440d48-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.614168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.614105    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-metrics-certs\") pod \"maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.715263 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.715173    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6chkj\" (UniqueName: \"kubernetes.io/projected/b1193647-74ea-437e-b1ae-9040d7440d48-kube-api-access-6chkj\") pod \"maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.715263 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.715247    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1193647-74ea-437e-b1ae-9040d7440d48-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.715501 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.715291    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-metrics-certs\") pod \"maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.715501 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.715363    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-maas-api-tls\") pod \"maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.715981 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.715957    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1193647-74ea-437e-b1ae-9040d7440d48-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.718004 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.717973    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-maas-api-tls\") pod \"maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.718004 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.717999    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-metrics-certs\") pod \"maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.741705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.741677    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6chkj\" (UniqueName: \"kubernetes.io/projected/b1193647-74ea-437e-b1ae-9040d7440d48-kube-api-access-6chkj\") pod \"maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.755528 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.755501    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:19.756518 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:19.756494    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:20.058378 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:20.058358    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"]
Oct 30 05:45:20.060023 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:45:20.059997    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1193647_74ea_437e_b1ae_9040d7440d48.slice/crio-ceef159985583528c6420406de04f30d86aed6cd44aa3f75ebc654e98ad458dd WatchSource:0}: Error finding container ceef159985583528c6420406de04f30d86aed6cd44aa3f75ebc654e98ad458dd: Status 404 returned error can't find the container with id ceef159985583528c6420406de04f30d86aed6cd44aa3f75ebc654e98ad458dd
Oct 30 05:45:20.521083 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:20.521050    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n" event={"ID":"b1193647-74ea-437e-b1ae-9040d7440d48","Type":"ContainerStarted","Data":"a545248539c988d3d44da4817b3b27d0ec83e824f0133edffebdda59bfb4841e"}
Oct 30 05:45:20.521493 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:20.521099    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n" event={"ID":"b1193647-74ea-437e-b1ae-9040d7440d48","Type":"ContainerStarted","Data":"ceef159985583528c6420406de04f30d86aed6cd44aa3f75ebc654e98ad458dd"}
Oct 30 05:45:20.521493 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:20.521154    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:20.625327 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:20.625271    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n" podStartSLOduration=1.293949218 podStartE2EDuration="1.625256219s" totalImagesPullingTime="331.307001ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:45:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:45:20.624246388 +0000 UTC m=+824.369531720" watchObservedRunningTime="2026-10-30 05:45:20.625256219 +0000 UTC m=+824.370541551"
Oct 30 05:45:21.398778 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:21.398737    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:23.122098 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:23.122064    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:24.520213 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:24.520182    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 05:45:24.947229 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:24.947199    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:25.150151 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:25.150118    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:25.326739 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:25.326655    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:26.432301 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.432273    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"]
Oct 30 05:45:26.436265 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.436248    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:26.468219 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.468187    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-128436-serving-cert\""
Oct 30 05:45:26.470249 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.470224    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"]
Oct 30 05:45:26.484885 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.484799    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mllj\" (UniqueName: \"kubernetes.io/projected/b64460f3-6d61-455e-923c-aa6e46ffca11-kube-api-access-6mllj\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:26.485054 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.484897    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64460f3-6d61-455e-923c-aa6e46ffca11-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:26.485054 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.484998    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-metrics-certs\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:26.485054 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.485023    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-maas-api-tls\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:26.533379 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.533347    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 05:45:26.577539 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.577500    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:26.586263 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.586237    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mllj\" (UniqueName: \"kubernetes.io/projected/b64460f3-6d61-455e-923c-aa6e46ffca11-kube-api-access-6mllj\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:26.586432 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.586287    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64460f3-6d61-455e-923c-aa6e46ffca11-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:26.586432 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.586374    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-metrics-certs\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:26.586432 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.586399    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-maas-api-tls\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:26.586932 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:45:26.586915    5066 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-serving-cert: secret "maas-api-e2e-worker-w3-128436-serving-cert" not found
Oct 30 05:45:26.587005 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:45:26.586987    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-maas-api-tls podName:b64460f3-6d61-455e-923c-aa6e46ffca11 nodeName:}" failed. No retries permitted until 2026-10-30 05:45:27.086971271 +0000 UTC m=+830.832256581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-maas-api-tls") pod "maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn" (UID: "b64460f3-6d61-455e-923c-aa6e46ffca11") : secret "maas-api-e2e-worker-w3-128436-serving-cert" not found
Oct 30 05:45:26.587694 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.587661    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64460f3-6d61-455e-923c-aa6e46ffca11-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:26.589959 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.589937    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-metrics-certs\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:26.612697 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.612669    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mllj\" (UniqueName: \"kubernetes.io/projected/b64460f3-6d61-455e-923c-aa6e46ffca11-kube-api-access-6mllj\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:26.734490 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.734407    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:26.891722 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:26.891693    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:27.073778 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:27.073695    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:27.091817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:27.091780    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-maas-api-tls\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:27.094401 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:27.094372    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-maas-api-tls\") pod \"maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:27.350368 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:27.350279    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:27.657445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:27.657414    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"]
Oct 30 05:45:27.659890 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:45:27.659863    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64460f3_6d61_455e_923c_aa6e46ffca11.slice/crio-27fd1554d6515d5d2f94ef408cd5be8afd04bbb77be827d35392686da977683b WatchSource:0}: Error finding container 27fd1554d6515d5d2f94ef408cd5be8afd04bbb77be827d35392686da977683b: Status 404 returned error can't find the container with id 27fd1554d6515d5d2f94ef408cd5be8afd04bbb77be827d35392686da977683b
Oct 30 05:45:28.336081 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:28.336001    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:28.564241 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:28.564205    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn" event={"ID":"b64460f3-6d61-455e-923c-aa6e46ffca11","Type":"ContainerStarted","Data":"c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d"}
Oct 30 05:45:28.564407 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:28.564267    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn" event={"ID":"b64460f3-6d61-455e-923c-aa6e46ffca11","Type":"ContainerStarted","Data":"27fd1554d6515d5d2f94ef408cd5be8afd04bbb77be827d35392686da977683b"}
Oct 30 05:45:28.564407 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:28.564331    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:28.611524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:28.611478    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn" podStartSLOduration=2.257209017 podStartE2EDuration="2.611464806s" totalImagesPullingTime="354.255789ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:45:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:45:28.610898377 +0000 UTC m=+832.356183708" watchObservedRunningTime="2026-10-30 05:45:28.611464806 +0000 UTC m=+832.356750178"
Oct 30 05:45:28.623281 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:28.623243    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:29.238754 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:29.238725    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:29.404385 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:29.404345    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:29.561399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:29.561324    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:30.026044 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.026010    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:30.304122 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.304022    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"]
Oct 30 05:45:30.312054 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.312023    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.330661 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.330632    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"]
Oct 30 05:45:30.342630 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.342601    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-c78066\"/\"openshift-service-ca.crt\""
Oct 30 05:45:30.342825 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.342668    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-c78066\"/\"kube-root-ca.crt\""
Oct 30 05:45:30.342825 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.342690    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c78066\"/\"default-dockercfg-bd499\""
Oct 30 05:45:30.342825 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.342763    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c78066\"/\"simulator-w2-c78066-kserve-self-signed-certs\""
Oct 30 05:45:30.422381 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.422347    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:30.426508 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.426474    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tls-certs\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.426675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.426539    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-model-cache\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.426780 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.426762    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkc29\" (UniqueName: \"kubernetes.io/projected/e98db60e-37c7-45cd-80d1-8770e15aaa0b-kube-api-access-zkc29\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.426830 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.426819    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-dshm\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.426911 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.426890    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-home\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.427072 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.427055    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tmp-dir\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.528153 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.528115    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tls-certs\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.528153 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.528152    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-model-cache\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.528387 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.528170    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkc29\" (UniqueName: \"kubernetes.io/projected/e98db60e-37c7-45cd-80d1-8770e15aaa0b-kube-api-access-zkc29\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.528387 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.528187    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-dshm\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.528387 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.528217    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-home\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.528387 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.528237    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tmp-dir\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.528635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.528579    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-model-cache\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.528682 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.528658    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tmp-dir\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.528726 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.528679    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-home\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.530805 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.530779    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-dshm\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.531055 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.531036    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tls-certs\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.557073 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.557003    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkc29\" (UniqueName: \"kubernetes.io/projected/e98db60e-37c7-45cd-80d1-8770e15aaa0b-kube-api-access-zkc29\") pod \"simulator-w2-c78066-kserve-66d4f5f87f-fc4n4\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") " pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.636421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.636386    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:30.972729 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:30.972705    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"]
Oct 30 05:45:30.974196 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:45:30.974175    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode98db60e_37c7_45cd_80d1_8770e15aaa0b.slice/crio-568a63422b5ca298429c68db4ce66867452fbe8b4043a5702662154d2236a8d2 WatchSource:0}: Error finding container 568a63422b5ca298429c68db4ce66867452fbe8b4043a5702662154d2236a8d2: Status 404 returned error can't find the container with id 568a63422b5ca298429c68db4ce66867452fbe8b4043a5702662154d2236a8d2
Oct 30 05:45:31.604177 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:31.604144    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" event={"ID":"e98db60e-37c7-45cd-80d1-8770e15aaa0b","Type":"ContainerStarted","Data":"568a63422b5ca298429c68db4ce66867452fbe8b4043a5702662154d2236a8d2"}
Oct 30 05:45:31.755634 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:31.755607    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:32.551960 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:45:32.551923    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:45:32.610830 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:32.610792    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" event={"ID":"e98db60e-37c7-45cd-80d1-8770e15aaa0b","Type":"ContainerStarted","Data":"9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e"}
Oct 30 05:45:32.637511 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:32.637468    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:32.657452 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:32.657396    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podStartSLOduration=1.4910204249999999 podStartE2EDuration="2.65738386s" totalImagesPullingTime="1.166363435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:45:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:45:32.655744574 +0000 UTC m=+836.401029906" watchObservedRunningTime="2026-10-30 05:45:32.65738386 +0000 UTC m=+836.402669186"
Oct 30 05:45:33.353632 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:33.353581    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:33.802028 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:33.801982    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:34.037849 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:34.037791    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:34.210976 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:34.210942    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:34.425903 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:34.425872    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:34.575828 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:34.575751    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 05:45:34.639034 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:34.638996    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:34.822005 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:34.821974    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:35.002956 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:35.002918    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:35.426914 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:35.426879    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:37.042019 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:37.041973    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:38.997373 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:38.997335    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:39.272894 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:39.272800    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:39.896411 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:39.896381    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:40.065037 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:40.064944    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:40.223153 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:40.223088    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:40.545542 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:40.545443    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:40.636783 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:40.636748    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:40.649790 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:40.649764    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:40.668288 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:40.668262    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:45:42.275816 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:42.275783    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:43.510441 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.510406    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"]
Oct 30 05:45:43.518142 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.518118    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.553141 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.553101    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-197e9b\"/\"kube-root-ca.crt\""
Oct 30 05:45:43.564837 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.564805    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-tmp-dir\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.564954 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.564868    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d2p4\" (UniqueName: \"kubernetes.io/projected/6f14e50e-4bde-4a37-838c-63b10269fa9e-kube-api-access-6d2p4\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.565014 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.564991    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-dshm\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.565075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.565027    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-model-cache\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.565139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.565115    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-home\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.565258 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.565235    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f14e50e-4bde-4a37-838c-63b10269fa9e-tls-certs\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.567289 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.567267    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-197e9b\"/\"openshift-service-ca.crt\""
Oct 30 05:45:43.576457 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.576431    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-197e9b\"/\"default-dockercfg-7rlx7\""
Oct 30 05:45:43.576457 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.576532    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-197e9b\"/\"simulator-w0-197e9b-kserve-self-signed-certs\""
Oct 30 05:45:43.583580 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.580134    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"]
Oct 30 05:45:43.665881 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.665846    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-dshm\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.665881 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.665881    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-model-cache\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.666149 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.665913    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-home\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.666149 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.665955    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f14e50e-4bde-4a37-838c-63b10269fa9e-tls-certs\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.666149 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.666089    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-tmp-dir\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.666149 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.666132    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d2p4\" (UniqueName: \"kubernetes.io/projected/6f14e50e-4bde-4a37-838c-63b10269fa9e-kube-api-access-6d2p4\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.666524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.666429    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-home\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.666688 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.666628    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-model-cache\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.666851 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.666786    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-tmp-dir\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.668617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.668565    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-dshm\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.669022 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.668987    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f14e50e-4bde-4a37-838c-63b10269fa9e-tls-certs\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.688565 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.688525    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:43.691724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.691704    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d2p4\" (UniqueName: \"kubernetes.io/projected/6f14e50e-4bde-4a37-838c-63b10269fa9e-kube-api-access-6d2p4\") pod \"simulator-w0-197e9b-kserve-b658b89f6-mmp4j\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") " pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:43.858161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:43.858074    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:44.161854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:44.161832    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"]
Oct 30 05:45:44.163441 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:45:44.163413    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f14e50e_4bde_4a37_838c_63b10269fa9e.slice/crio-41201c01e46d9befd94ab9c934510c6bbb7e75a93dda96982acc499fb0677e28 WatchSource:0}: Error finding container 41201c01e46d9befd94ab9c934510c6bbb7e75a93dda96982acc499fb0677e28: Status 404 returned error can't find the container with id 41201c01e46d9befd94ab9c934510c6bbb7e75a93dda96982acc499fb0677e28
Oct 30 05:45:44.224400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:44.224370    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:44.681043 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:44.681000    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" event={"ID":"6f14e50e-4bde-4a37-838c-63b10269fa9e","Type":"ContainerStarted","Data":"8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02"}
Oct 30 05:45:44.681527 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:44.681053    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" event={"ID":"6f14e50e-4bde-4a37-838c-63b10269fa9e","Type":"ContainerStarted","Data":"41201c01e46d9befd94ab9c934510c6bbb7e75a93dda96982acc499fb0677e28"}
Oct 30 05:45:44.728660 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:44.728608    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podStartSLOduration=1.728572371 podStartE2EDuration="1.728572371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:45:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:45:44.726530217 +0000 UTC m=+848.471815547" watchObservedRunningTime="2026-10-30 05:45:44.728572371 +0000 UTC m=+848.473857701"
Oct 30 05:45:44.858701 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:44.858672    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:45.262107 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:45.262081    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:45.463375 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:45.463335    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:45.741554 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:45.741517    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:45.926548 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:45.926518    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:46.104071 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.103993    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:46.310048 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.310012    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:46.618266 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.618196    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"]
Oct 30 05:45:46.623126 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.623089    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.657678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.657642    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e2e3d0\"/\"default-dockercfg-l9xmp\""
Oct 30 05:45:46.658095 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.658070    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e2e3d0\"/\"simulator-w4-e2e3d0-kserve-self-signed-certs\""
Oct 30 05:45:46.658245 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.658178    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e2e3d0\"/\"openshift-service-ca.crt\""
Oct 30 05:45:46.658317 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.658269    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e2e3d0\"/\"kube-root-ca.crt\""
Oct 30 05:45:46.661474 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.661449    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"]
Oct 30 05:45:46.695331 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.695297    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-dshm\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.695526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.695368    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-model-cache\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.695526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.695492    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef3506e3-2464-46ed-8808-3c5d681330ad-tls-certs\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.695680 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.695550    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-tmp-dir\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.695680 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.695619    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-home\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.695680 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.695647    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb2vh\" (UniqueName: \"kubernetes.io/projected/ef3506e3-2464-46ed-8808-3c5d681330ad-kube-api-access-lb2vh\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.738209 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.738177    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:46.796338 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.796295    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-dshm\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.796869 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.796373    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-model-cache\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.796869 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.796414    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef3506e3-2464-46ed-8808-3c5d681330ad-tls-certs\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.796869 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.796490    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-tmp-dir\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.796869 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.796571    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-home\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.796869 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.796649    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb2vh\" (UniqueName: \"kubernetes.io/projected/ef3506e3-2464-46ed-8808-3c5d681330ad-kube-api-access-lb2vh\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.797317 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.796919    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-model-cache\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.797446 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.797423    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-home\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.797676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.797655    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-tmp-dir\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.798897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.798875    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-dshm\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.799384 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.799363    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef3506e3-2464-46ed-8808-3c5d681330ad-tls-certs\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.826894 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.826863    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb2vh\" (UniqueName: \"kubernetes.io/projected/ef3506e3-2464-46ed-8808-3c5d681330ad-kube-api-access-lb2vh\") pod \"simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") " pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:46.936896 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:46.936848    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:47.296060 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:47.296029    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"]
Oct 30 05:45:47.304706 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:45:47.304673    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef3506e3_2464_46ed_8808_3c5d681330ad.slice/crio-9e0930cfda20c51b713fb55cfcff01327089f89f084525f9e6d156170104cb1c WatchSource:0}: Error finding container 9e0930cfda20c51b713fb55cfcff01327089f89f084525f9e6d156170104cb1c: Status 404 returned error can't find the container with id 9e0930cfda20c51b713fb55cfcff01327089f89f084525f9e6d156170104cb1c
Oct 30 05:45:47.698413 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:47.698380    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" event={"ID":"ef3506e3-2464-46ed-8808-3c5d681330ad","Type":"ContainerStarted","Data":"cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f"}
Oct 30 05:45:47.698413 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:47.698419    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" event={"ID":"ef3506e3-2464-46ed-8808-3c5d681330ad","Type":"ContainerStarted","Data":"9e0930cfda20c51b713fb55cfcff01327089f89f084525f9e6d156170104cb1c"}
Oct 30 05:45:47.746785 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:47.746736    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podStartSLOduration=1.746721128 podStartE2EDuration="1.746721128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:45:47.745567416 +0000 UTC m=+851.490852744" watchObservedRunningTime="2026-10-30 05:45:47.746721128 +0000 UTC m=+851.492006458"
Oct 30 05:45:47.748235 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:47.748216    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:47.937482 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:47.937447    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:48.253382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:48.253331    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:49.501067 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:49.501031    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:49.869918 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:49.869829    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"]
Oct 30 05:45:49.874580 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:49.874554    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:49.897276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:49.897242    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"]
Oct 30 05:45:49.908909 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:49.908881    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-4a68a1\"/\"openshift-service-ca.crt\""
Oct 30 05:45:49.909133 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:49.908880    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-4a68a1\"/\"kube-root-ca.crt\""
Oct 30 05:45:49.909133 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:49.909006    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-4a68a1\"/\"simulator-w1-4a68a1-kserve-self-signed-certs\""
Oct 30 05:45:49.909297 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:49.909060    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-4a68a1\"/\"default-dockercfg-576t9\""
Oct 30 05:45:49.977388 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:49.977349    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:50.032880 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.032837    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-dshm\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.033036 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.032900    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-home\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.033135 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.033103    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-model-cache\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.033258 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.033240    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5edbb8c6-de2c-4167-8e45-ba6c78204165-tls-certs\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.033310 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.033294    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84snr\" (UniqueName: \"kubernetes.io/projected/5edbb8c6-de2c-4167-8e45-ba6c78204165-kube-api-access-84snr\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.033409 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.033391    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-tmp-dir\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.134946 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.134848    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-dshm\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.134946 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.134908    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-home\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.135190 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.134982    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-model-cache\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.135190 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.135019    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5edbb8c6-de2c-4167-8e45-ba6c78204165-tls-certs\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.135190 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.135049    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84snr\" (UniqueName: \"kubernetes.io/projected/5edbb8c6-de2c-4167-8e45-ba6c78204165-kube-api-access-84snr\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.135190 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.135070    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-tmp-dir\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.135420 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.135393    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-home\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.135474 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.135427    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-model-cache\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.135507 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.135486    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-tmp-dir\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.137269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.137247    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-dshm\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.137581 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.137563    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5edbb8c6-de2c-4167-8e45-ba6c78204165-tls-certs\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.160277 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.160247    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84snr\" (UniqueName: \"kubernetes.io/projected/5edbb8c6-de2c-4167-8e45-ba6c78204165-kube-api-access-84snr\") pod \"simulator-w1-4a68a1-kserve-754678b89c-dkwgp\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") " pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.188364 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.188303    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:50.501475 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.501447    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"]
Oct 30 05:45:50.503193 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:45:50.503166    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5edbb8c6_de2c_4167_8e45_ba6c78204165.slice/crio-646b3958104e0add76bf244d7e6dbe50ee8321a6545df6a14983c40cb9f0207c WatchSource:0}: Error finding container 646b3958104e0add76bf244d7e6dbe50ee8321a6545df6a14983c40cb9f0207c: Status 404 returned error can't find the container with id 646b3958104e0add76bf244d7e6dbe50ee8321a6545df6a14983c40cb9f0207c
Oct 30 05:45:50.714967 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.714930    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" event={"ID":"5edbb8c6-de2c-4167-8e45-ba6c78204165","Type":"ContainerStarted","Data":"539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031"}
Oct 30 05:45:50.714967 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.714971    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" event={"ID":"5edbb8c6-de2c-4167-8e45-ba6c78204165","Type":"ContainerStarted","Data":"646b3958104e0add76bf244d7e6dbe50ee8321a6545df6a14983c40cb9f0207c"}
Oct 30 05:45:50.765303 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:50.765187    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podStartSLOduration=1.765150437 podStartE2EDuration="1.765150437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:45:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:45:50.763754391 +0000 UTC m=+854.509039722" watchObservedRunningTime="2026-10-30 05:45:50.765150437 +0000 UTC m=+854.510435768"
Oct 30 05:45:51.128557 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:51.128456    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:51.189112 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:51.189079    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:45:52.822447 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:52.822401    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:53.858501 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:53.858467    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:53.871318 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:53.871299    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:54.438335 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:54.438288    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:54.744980 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:54.744904    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:45:56.413261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:56.413232    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:56.937306 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:56.937266    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:56.950547 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:56.950518    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:57.761094 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:57.761068    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:45:57.993905 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:57.993877    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:45:59.584743 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:45:59.584708    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:00.189551 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:00.189509    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:46:00.202375 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:00.202348    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:46:00.778645 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:00.778616    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:46:01.434736 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:01.434688    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:02.289284 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:02.289250    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:02.551851 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:46:02.551724    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:46:03.191742 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:03.191705    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:04.491198 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:04.491156    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:04.838997 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:04.838905    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:05.002693 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:05.002661    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:06.325715 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.325677    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"]
Oct 30 05:46:06.330287 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.330261    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.350427 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.350395    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"]
Oct 30 05:46:06.361901 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.361876    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-1354a5\"/\"kube-root-ca.crt\""
Oct 30 05:46:06.361901 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.361875    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-1354a5\"/\"openshift-service-ca.crt\""
Oct 30 05:46:06.362131 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.362020    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-1354a5\"/\"default-dockercfg-cnhtk\""
Oct 30 05:46:06.362131 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.362080    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-1354a5\"/\"simulator-w6-1354a5-kserve-self-signed-certs\""
Oct 30 05:46:06.399270 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.399236    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-model-cache\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.399451 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.399287    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tmp-dir\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.399451 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.399323    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-dshm\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.399451 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.399390    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tls-certs\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.399565 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.399486    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-home\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.399565 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.399544    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbgff\" (UniqueName: \"kubernetes.io/projected/71a86815-9e7e-4c32-a865-e51a1c51c7c9-kube-api-access-bbgff\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.434611 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.434564    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:06.500740 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.500703    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-model-cache\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.500740 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.500748    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tmp-dir\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.501005 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.500772    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-dshm\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.501005 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.500794    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tls-certs\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.501005 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.500842    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-home\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.501005 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.500907    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbgff\" (UniqueName: \"kubernetes.io/projected/71a86815-9e7e-4c32-a865-e51a1c51c7c9-kube-api-access-bbgff\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.501232 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.501181    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tmp-dir\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.501232 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.501197    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-model-cache\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.501410 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.501388    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-home\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.503224 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.503204    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-dshm\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.503852 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.503837    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tls-certs\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.526726 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.526693    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbgff\" (UniqueName: \"kubernetes.io/projected/71a86815-9e7e-4c32-a865-e51a1c51c7c9-kube-api-access-bbgff\") pod \"simulator-w6-1354a5-kserve-86d8859b-mvdrn\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") " pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.595480 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.595388    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:06.643304 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.643274    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:06.976979 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:06.976951    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"]
Oct 30 05:46:06.978781 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:46:06.978753    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71a86815_9e7e_4c32_a865_e51a1c51c7c9.slice/crio-d69c83f746414e513dc059273df766ae36621e9655b2c19411fc6ea75dbf8d73 WatchSource:0}: Error finding container d69c83f746414e513dc059273df766ae36621e9655b2c19411fc6ea75dbf8d73: Status 404 returned error can't find the container with id d69c83f746414e513dc059273df766ae36621e9655b2c19411fc6ea75dbf8d73
Oct 30 05:46:07.802791 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:07.802750    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" event={"ID":"71a86815-9e7e-4c32-a865-e51a1c51c7c9","Type":"ContainerStarted","Data":"ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd"}
Oct 30 05:46:07.803221 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:07.802802    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" event={"ID":"71a86815-9e7e-4c32-a865-e51a1c51c7c9","Type":"ContainerStarted","Data":"d69c83f746414e513dc059273df766ae36621e9655b2c19411fc6ea75dbf8d73"}
Oct 30 05:46:07.849923 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:07.849865    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podStartSLOduration=1.849847343 podStartE2EDuration="1.849847343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:46:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:46:07.847237647 +0000 UTC m=+871.592522979" watchObservedRunningTime="2026-10-30 05:46:07.849847343 +0000 UTC m=+871.595132670"
Oct 30 05:46:08.139161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:08.139060    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:08.643681 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:08.643646    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:09.729927 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:09.729887    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:11.471744 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:11.471708    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:13.201559 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:13.201524    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:14.934564 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:14.934532    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:15.353402 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:15.353315    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:16.577939 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:16.577908    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:16.643487 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:16.643453    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:16.656279 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:16.656249    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:16.860901 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:16.860815    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:46:18.280799 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:18.280769    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:20.046432 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:20.046398    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:21.449280 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:21.449247    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:21.680733 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:21.680702    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:23.425272 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:23.425231    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:24.911390 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:24.911352    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:26.458790 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:26.458759    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:27.196857 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:27.196827    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:28.074871 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.074834    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:28.266253 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.266219    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"]
Oct 30 05:46:28.270175 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.270153    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.303010 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.302980    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-4a68a1\"/\"premium-w1-4a68a1-kserve-self-signed-certs\""
Oct 30 05:46:28.305144 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.305123    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"]
Oct 30 05:46:28.441666 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.441628    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-dshm\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.441882 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.441715    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-model-cache\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.441882 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.441737    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-tmp-dir\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.441882 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.441807    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5d6f936-0eb8-4937-b596-497662f44fbc-tls-certs\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.442062 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.441907    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xznsw\" (UniqueName: \"kubernetes.io/projected/a5d6f936-0eb8-4937-b596-497662f44fbc-kube-api-access-xznsw\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.442062 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.441935    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-home\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.543140 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.543098    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xznsw\" (UniqueName: \"kubernetes.io/projected/a5d6f936-0eb8-4937-b596-497662f44fbc-kube-api-access-xznsw\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.543140 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.543142    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-home\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.543378 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.543181    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-dshm\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.543378 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.543227    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-model-cache\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.543378 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.543242    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-tmp-dir\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.543378 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.543279    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5d6f936-0eb8-4937-b596-497662f44fbc-tls-certs\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.543822 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.543784    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-tmp-dir\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.543822 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.543818    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-model-cache\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.544026 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.543943    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-home\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.545794 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.545774    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-dshm\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.546445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.546418    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5d6f936-0eb8-4937-b596-497662f44fbc-tls-certs\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.568602 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.568556    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xznsw\" (UniqueName: \"kubernetes.io/projected/a5d6f936-0eb8-4937-b596-497662f44fbc-kube-api-access-xznsw\") pod \"premium-w1-4a68a1-kserve-596d74cf4d-cmc9g\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") " pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.586431 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.586394    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:28.898754 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.898717    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"]
Oct 30 05:46:28.899391 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:46:28.899359    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5d6f936_0eb8_4937_b596_497662f44fbc.slice/crio-41382d7ac4af739f5385abfda1b95cbbc3b43cddc51ad486618124e0576e2585 WatchSource:0}: Error finding container 41382d7ac4af739f5385abfda1b95cbbc3b43cddc51ad486618124e0576e2585: Status 404 returned error can't find the container with id 41382d7ac4af739f5385abfda1b95cbbc3b43cddc51ad486618124e0576e2585
Oct 30 05:46:28.913323 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:28.913248    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" event={"ID":"a5d6f936-0eb8-4937-b596-497662f44fbc","Type":"ContainerStarted","Data":"41382d7ac4af739f5385abfda1b95cbbc3b43cddc51ad486618124e0576e2585"}
Oct 30 05:46:29.921953 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:29.921916    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" event={"ID":"a5d6f936-0eb8-4937-b596-497662f44fbc","Type":"ContainerStarted","Data":"68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b"}
Oct 30 05:46:29.969566 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:29.969504    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podStartSLOduration=1.969483013 podStartE2EDuration="1.969483013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:46:29.967338608 +0000 UTC m=+893.712623941" watchObservedRunningTime="2026-10-30 05:46:29.969483013 +0000 UTC m=+893.714768345"
Oct 30 05:46:30.109665 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:30.109575    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:30.587324 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:30.587290    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:30.636082 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:30.636053    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:31.581007 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:31.580973    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:32.552264 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:46:32.552229    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:46:33.216865 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:33.216829    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:34.268630 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:34.268575    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:35.023633 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:35.023600    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:36.482704 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:36.482673    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:46:36.484698 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:36.484678    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:46:36.630981 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:36.630946    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:38.237199 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.237162    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:38.390416 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.390385    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"]
Oct 30 05:46:38.395694 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.395669    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.429949 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.429906    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-128436\"/\"kube-root-ca.crt\""
Oct 30 05:46:38.429949 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.429941    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-128436\"/\"openshift-service-ca.crt\""
Oct 30 05:46:38.430252 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.430236    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-128436\"/\"premium-w3-128436-kserve-self-signed-certs\""
Oct 30 05:46:38.430388 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.430371    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-128436\"/\"default-dockercfg-4hqd7\""
Oct 30 05:46:38.432016 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.431994    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"]
Oct 30 05:46:38.544170 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.544082    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-model-cache\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.544170 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.544140    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnq4q\" (UniqueName: \"kubernetes.io/projected/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-kube-api-access-bnq4q\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.544368 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.544274    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-dshm\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.544368 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.544313    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-home\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.544368 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.544346    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tmp-dir\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.544523 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.544507    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tls-certs\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.587474 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.587441    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:38.604056 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.604029    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:38.645855 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.645804    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-model-cache\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.646055 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.645901    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnq4q\" (UniqueName: \"kubernetes.io/projected/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-kube-api-access-bnq4q\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.646055 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.645944    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-dshm\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.646055 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.645976    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-home\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.646055 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.646005    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tmp-dir\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.646264 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.646121    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tls-certs\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.646578 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.646556    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-model-cache\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.646700 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.646659    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-home\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.646901 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.646881    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tmp-dir\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.649387 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.649364    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-dshm\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.649844 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.649822    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tls-certs\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.660911 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.660854    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:38.676152 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.676123    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnq4q\" (UniqueName: \"kubernetes.io/projected/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-kube-api-access-bnq4q\") pod \"premium-w3-128436-kserve-85bd66c7c-7njnl\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") " pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.712496 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.711251    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:38.982704 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:38.982669    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:46:39.038329 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:39.038303    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"]
Oct 30 05:46:39.040208 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:46:39.040175    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode74e2be3_9336_4f70_81bf_e7caba8d1a8d.slice/crio-5ad1256478d3b999c8545db1e1ecf015f0daa88d4513f6a1784f28b8163a8a9f WatchSource:0}: Error finding container 5ad1256478d3b999c8545db1e1ecf015f0daa88d4513f6a1784f28b8163a8a9f: Status 404 returned error can't find the container with id 5ad1256478d3b999c8545db1e1ecf015f0daa88d4513f6a1784f28b8163a8a9f
Oct 30 05:46:39.978962 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:39.978919    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" event={"ID":"e74e2be3-9336-4f70-81bf-e7caba8d1a8d","Type":"ContainerStarted","Data":"7192f1a06a3553c4364aee95a0a9eef78ed8ff48099d351d40dd583a6f556f35"}
Oct 30 05:46:39.978962 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:39.978974    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" event={"ID":"e74e2be3-9336-4f70-81bf-e7caba8d1a8d","Type":"ContainerStarted","Data":"5ad1256478d3b999c8545db1e1ecf015f0daa88d4513f6a1784f28b8163a8a9f"}
Oct 30 05:46:40.028806 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:40.028759    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podStartSLOduration=2.02874287 podStartE2EDuration="2.02874287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:46:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:46:40.02765866 +0000 UTC m=+903.772943992" watchObservedRunningTime="2026-10-30 05:46:40.02874287 +0000 UTC m=+903.774028206"
Oct 30 05:46:40.132490 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:40.132457    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:40.711469 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:40.711440    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:41.323286 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:41.323252    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:41.483791 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:41.483754    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:41.714891 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:41.714859    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:43.234529 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:43.234487    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:44.974703 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:44.974666    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:46.775477 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:46.775447    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:48.269406 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:48.269328    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:48.712261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:48.712223    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:48.725521 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:48.725495    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:49.036467 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:49.036380    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:46:50.109828 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:50.109781    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:50.953943 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:50.953911    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"]
Oct 30 05:46:50.957987 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:50.957963    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:50.994259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:50.994231    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-1354a5\"/\"premium-w6-1354a5-kserve-self-signed-certs\""
Oct 30 05:46:50.996712 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:50.996685    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"]
Oct 30 05:46:51.077543 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.077509    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-dshm\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.077754 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.077549    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5qh8\" (UniqueName: \"kubernetes.io/projected/40749c08-d2c2-4a79-acb6-45b23d69f7db-kube-api-access-s5qh8\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.077835 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.077742    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-tmp-dir\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.077898 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.077849    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-model-cache\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.077898 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.077887    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-home\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.078006 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.077953    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40749c08-d2c2-4a79-acb6-45b23d69f7db-tls-certs\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.139478 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.139443    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:51.178989 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.178956    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-dshm\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.179153 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.179000    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5qh8\" (UniqueName: \"kubernetes.io/projected/40749c08-d2c2-4a79-acb6-45b23d69f7db-kube-api-access-s5qh8\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.179153 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.179060    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-tmp-dir\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.179153 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.179142    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-model-cache\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.179260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.179166    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-home\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.179260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.179204    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40749c08-d2c2-4a79-acb6-45b23d69f7db-tls-certs\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.179705 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.179640    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-home\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.180242 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.180220    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-model-cache\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.180455 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.180437    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-tmp-dir\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.181526 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.181504    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-dshm\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.181722 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.181701    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40749c08-d2c2-4a79-acb6-45b23d69f7db-tls-certs\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.206252 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.206186    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5qh8\" (UniqueName: \"kubernetes.io/projected/40749c08-d2c2-4a79-acb6-45b23d69f7db-kube-api-access-s5qh8\") pod \"premium-w6-1354a5-kserve-6659874989-hw89h\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") " pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.271858 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.271813    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:51.587858 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:51.587832    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"]
Oct 30 05:46:51.589702 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:46:51.589674    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40749c08_d2c2_4a79_acb6_45b23d69f7db.slice/crio-71adaa4aae37c795fb415596ab7acf31317aaaeccbadf4cfa5783094995b5345 WatchSource:0}: Error finding container 71adaa4aae37c795fb415596ab7acf31317aaaeccbadf4cfa5783094995b5345: Status 404 returned error can't find the container with id 71adaa4aae37c795fb415596ab7acf31317aaaeccbadf4cfa5783094995b5345
Oct 30 05:46:52.045856 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:52.045826    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" event={"ID":"40749c08-d2c2-4a79-acb6-45b23d69f7db","Type":"ContainerStarted","Data":"e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e"}
Oct 30 05:46:52.045856 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:52.045862    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" event={"ID":"40749c08-d2c2-4a79-acb6-45b23d69f7db","Type":"ContainerStarted","Data":"71adaa4aae37c795fb415596ab7acf31317aaaeccbadf4cfa5783094995b5345"}
Oct 30 05:46:52.063366 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:52.063337    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:52.214839 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:52.214782    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podStartSLOduration=2.214765153 podStartE2EDuration="2.214765153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:46:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:46:52.203990577 +0000 UTC m=+915.949275911" watchObservedRunningTime="2026-10-30 05:46:52.214765153 +0000 UTC m=+915.960050484"
Oct 30 05:46:52.272637 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:52.272564    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:46:54.068221 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:54.068167    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:55.667791 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:55.667755    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:57.229894 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:57.229859    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:46:59.194747 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:46:59.194711    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:01.065760 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:01.065724    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:01.273287 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:01.273250    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:47:01.287443 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:01.287415    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:47:02.111605 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:02.111561    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:47:02.555433 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:47:02.555394    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:47:02.953431 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:02.953400    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:04.769077 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:04.769049    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:07.004707 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:07.004672    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:08.815650 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:08.815582    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:10.511709 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:10.511675    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:12.119439 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:12.119385    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:13.894624 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:13.894559    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:15.560380 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:15.560305    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:17.251724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:17.251650    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:18.958262 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:18.958234    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:19.965893 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:19.965863    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"]
Oct 30 05:47:19.969479 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:19.969459    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:19.992787 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:19.992755    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-data\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:19.992940 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:19.992804    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-certs\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:19.992940 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:19.992828    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-podinfo\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:19.992940 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:19.992894    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-token\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:19.993261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:19.993051    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4c64\" (UniqueName: \"kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-kube-api-access-l4c64\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:19.993261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:19.993131    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istiod-ca-cert\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:19.993261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:19.993166    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:19.993261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:19.993212    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-credential-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:19.993261 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:19.993245    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-envoy\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:19.996504 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:19.996484    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"]
Oct 30 05:47:20.007817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.007793    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-4a68a1-openshift-default-dockercfg-mbxx7\""
Oct 30 05:47:20.094833 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.094794    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istiod-ca-cert\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.095021 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.094852    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.095021 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.094901    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-credential-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.095021 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.094937    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-envoy\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.095021 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.095005    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-data\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.095259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.095053    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-certs\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.095259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.095078    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-podinfo\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.095259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.095103    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-token\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.095259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.095217    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4c64\" (UniqueName: \"kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-kube-api-access-l4c64\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.095472 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.095450    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-credential-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.095567 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.095547    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-certs\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.095954 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.095921    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istiod-ca-cert\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.096215 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.096032    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.096479 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.096452    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-data\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.097790 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.097767    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-envoy\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.098067 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.098029    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-podinfo\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.121477 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.121449    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-token\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.121613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.121513    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4c64\" (UniqueName: \"kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-kube-api-access-l4c64\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.284950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.284869    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:20.613979 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.613946    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"]
Oct 30 05:47:20.615385 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:47:20.615356    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd345cc1b_21b6_44b4_9ca6_3a5c0acf93b7.slice/crio-5267a353fbe1df7f2c030b9cd00148a29f9bdbdcbb5089fa12f7dd584a6a4e04 WatchSource:0}: Error finding container 5267a353fbe1df7f2c030b9cd00148a29f9bdbdcbb5089fa12f7dd584a6a4e04: Status 404 returned error can't find the container with id 5267a353fbe1df7f2c030b9cd00148a29f9bdbdcbb5089fa12f7dd584a6a4e04
Oct 30 05:47:20.617821 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.617789    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:47:20.617918 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.617871    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:47:20.617985 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.617915    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:47:20.894997 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:20.894923    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:21.203676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:21.203637    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" event={"ID":"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7","Type":"ContainerStarted","Data":"0b1aae5feea16897fd3b9cd007d66c48a510cd04dd4fa91ceaf5fba82e935c99"}
Oct 30 05:47:21.204054 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:21.203726    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" event={"ID":"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7","Type":"ContainerStarted","Data":"5267a353fbe1df7f2c030b9cd00148a29f9bdbdcbb5089fa12f7dd584a6a4e04"}
Oct 30 05:47:21.267135 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:21.267085    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" podStartSLOduration=2.267070502 podStartE2EDuration="2.267070502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:47:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:47:21.266055698 +0000 UTC m=+945.011341032" watchObservedRunningTime="2026-10-30 05:47:21.267070502 +0000 UTC m=+945.012355834"
Oct 30 05:47:21.285875 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:21.285835    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:21.287228 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:21.287204    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 05:47:21.287350 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:21.287247    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 05:47:22.285188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:22.285155    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 05:47:22.285568 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:22.285205    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 05:47:22.529896 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:22.529860    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:23.285668 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:23.285636    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 05:47:23.286058 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:23.285686    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 05:47:24.286098 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:24.286067    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 05:47:24.286484 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:24.286117    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 05:47:24.355144 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:24.355100    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:25.285205 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:25.285174    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 05:47:25.285390 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:25.285226    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 05:47:26.203217 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:26.203187    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:26.285525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:26.285493    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 05:47:26.285706 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:26.285547    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 05:47:27.285937 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:27.285891    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 05:47:27.286374 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:27.285967    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 05:47:27.324629 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:27.324575    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:28.171612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:28.171555    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:28.376225 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:28.376196    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:28.377018 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:28.376995    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:28.377862 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:28.377841    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 05:47:29.926035 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:29.925998    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:31.743542 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:31.743508    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:32.552008 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:47:32.551977    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:47:33.740002 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:33.739939    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:35.657514 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.657481    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"]
Oct 30 05:47:35.663029 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.663008    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.688904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.688876    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-e2e3d0-openshift-default-dockercfg-dkznd\""
Oct 30 05:47:35.698052 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.698022    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"]
Oct 30 05:47:35.706702 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.706663    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:35.794794 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.794758    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-data\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.794962 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.794801    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-certs\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.795004 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.794970    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-envoy\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.795004 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.794999    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-istio-token\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.795068 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.795025    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-credential-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.795150 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.795132    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dafc9b85-538c-4f66-8578-012bad182b0a-istio-podinfo\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.795188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.795161    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.795221 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.795195    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz48z\" (UniqueName: \"kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-kube-api-access-dz48z\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.795254 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.795246    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dafc9b85-538c-4f66-8578-012bad182b0a-istiod-ca-cert\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.896849 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.896817    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-credential-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.897044 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.896868    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dafc9b85-538c-4f66-8578-012bad182b0a-istio-podinfo\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.897044 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.896887    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.897044 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.896923    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz48z\" (UniqueName: \"kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-kube-api-access-dz48z\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.897044 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.896956    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dafc9b85-538c-4f66-8578-012bad182b0a-istiod-ca-cert\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.897044 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.896990    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-data\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.897044 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.897027    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-certs\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.897376 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.897188    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-envoy\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.897376 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.897220    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-istio-token\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.897376 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.897314    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-credential-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.897527 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.897501    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-data\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.897622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.897502    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-certs\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.897682 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.897623    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.898241 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.898215    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dafc9b85-538c-4f66-8578-012bad182b0a-istiod-ca-cert\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.899582 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.899559    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-envoy\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.899689 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.899653    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dafc9b85-538c-4f66-8578-012bad182b0a-istio-podinfo\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.928402 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.928331    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-istio-token\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.928505 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.928426    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz48z\" (UniqueName: \"kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-kube-api-access-dz48z\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:35.980647 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:35.980619    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:36.310177 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:36.310084    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"]
Oct 30 05:47:36.315657 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:47:36.315631    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddafc9b85_538c_4f66_8578_012bad182b0a.slice/crio-537c7a025a41e0b6668127eac9d2c095f5a0e6c916c6443578b318ecfccc9508 WatchSource:0}: Error finding container 537c7a025a41e0b6668127eac9d2c095f5a0e6c916c6443578b318ecfccc9508: Status 404 returned error can't find the container with id 537c7a025a41e0b6668127eac9d2c095f5a0e6c916c6443578b318ecfccc9508
Oct 30 05:47:36.317784 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:36.317750    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:47:36.317877 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:36.317815    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:47:36.317877 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:36.317842    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:47:37.319293 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:37.319261    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" event={"ID":"dafc9b85-538c-4f66-8578-012bad182b0a","Type":"ContainerStarted","Data":"d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849"}
Oct 30 05:47:37.319820 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:37.319318    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" event={"ID":"dafc9b85-538c-4f66-8578-012bad182b0a","Type":"ContainerStarted","Data":"537c7a025a41e0b6668127eac9d2c095f5a0e6c916c6443578b318ecfccc9508"}
Oct 30 05:47:37.397149 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:37.397093    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" podStartSLOduration=2.397080044 podStartE2EDuration="2.397080044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:47:37.395979244 +0000 UTC m=+961.141264575" watchObservedRunningTime="2026-10-30 05:47:37.397080044 +0000 UTC m=+961.142365375"
Oct 30 05:47:37.471454 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:37.471425    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:37.980979 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:37.980941    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:37.982695 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:37.982664    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 05:47:37.982818 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:37.982718    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" podUID="dafc9b85-538c-4f66-8578-012bad182b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 05:47:39.016139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:39.016096    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 05:47:39.016542 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:39.016174    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" podUID="dafc9b85-538c-4f66-8578-012bad182b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 05:47:39.394275 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:39.393949    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:39.981997 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:39.981921    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 05:47:39.981997 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:39.981989    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" podUID="dafc9b85-538c-4f66-8578-012bad182b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 05:47:40.982617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:40.982212    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 05:47:40.982617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:40.982269    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" podUID="dafc9b85-538c-4f66-8578-012bad182b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 05:47:41.360673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:41.359705    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xfps7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:47:41.360673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:41.359773    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:47:41.422172 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:41.422136    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:41.623826 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:41.623721    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-p99cc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:47:41.624227 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:41.624179    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:47:41.672754 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:41.672714    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xjhtt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:47:41.673081 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:41.673032    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:47:41.988453 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:41.988419    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 05:47:41.989083 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:41.989054    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" podUID="dafc9b85-538c-4f66-8578-012bad182b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 05:47:43.255539 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:43.255498    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:43.256125 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:43.255846    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:43.256648 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:43.256630    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 05:47:43.574448 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:43.574371    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:45.469295 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:45.469261    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:47.451729 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:47.451697    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:47.631695 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:47.631663    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:49.127612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:49.127567    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:50.808272 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:50.808236    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:52.356659 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:52.356625    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:53.826885 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:53.826853    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:55.612616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:55.612564    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:56.615773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:56.615734    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-p99cc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:47:56.616275 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:56.615807    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:47:56.887122 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:56.887040    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:57.305510 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:57.305481    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:47:59.009748 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:47:59.009711    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:00.588339 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:00.588303    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:02.114005 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:02.113973    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:02.551317 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:48:02.551281    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:48:03.955400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:03.955371    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:05.095993 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.095953    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"]
Oct 30 05:48:05.104666 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.102685    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.128689 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.128660    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"]
Oct 30 05:48:05.263371 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.263336    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e09beacd-ed50-4e0d-b633-8900fae01778-istiod-ca-cert\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.263536 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.263391    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-credential-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.263536 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.263432    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e09beacd-ed50-4e0d-b633-8900fae01778-istio-podinfo\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.263536 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.263523    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-certs\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.263676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.263634    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.263676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.263659    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klrh5\" (UniqueName: \"kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-kube-api-access-klrh5\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.263742 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.263683    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-data\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.263775 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.263743    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-istio-token\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.263834 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.263809    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-envoy\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.300608 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.300553    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"]
Oct 30 05:48:05.304671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.304650    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.330161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.330135    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-197e9b-openshift-default-dockercfg-w5q2q\""
Oct 30 05:48:05.335658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.335628    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"]
Oct 30 05:48:05.364496 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.364411    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e09beacd-ed50-4e0d-b633-8900fae01778-istio-podinfo\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.364496 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.364453    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-certs\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.364496 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.364495    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.364835 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.364518    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klrh5\" (UniqueName: \"kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-kube-api-access-klrh5\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.364835 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.364557    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-data\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.364835 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.364618    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-istio-token\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.364835 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.364650    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-envoy\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.364835 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.364752    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e09beacd-ed50-4e0d-b633-8900fae01778-istiod-ca-cert\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.364835 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.364828    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-credential-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.365156 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.365028    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.365217 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.365194    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-credential-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.365726 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.365671    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-data\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.366109 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.366088    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-certs\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.366500 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.366464    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e09beacd-ed50-4e0d-b633-8900fae01778-istiod-ca-cert\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.367748 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.367726    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-envoy\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.368063 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.368043    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e09beacd-ed50-4e0d-b633-8900fae01778-istio-podinfo\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.393441 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.393402    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-istio-token\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.395238 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.395214    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klrh5\" (UniqueName: \"kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-kube-api-access-klrh5\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.421789 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.421764    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:05.466080 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.466044    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-podinfo\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.466286 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.466145    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-envoy\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.466286 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.466262    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/576c2b0b-6909-4f31-99b5-1dd353650d0d-istiod-ca-cert\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.466401 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.466317    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-certs\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.466401 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.466342    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-token\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.466401 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.466360    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhfjw\" (UniqueName: \"kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-kube-api-access-fhfjw\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.466573 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.466541    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.466697 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.466667    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-credential-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.466771 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.466703    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-data\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.550131 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.550098    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:05.568419 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.568384    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-envoy\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.568419 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.568418    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/576c2b0b-6909-4f31-99b5-1dd353650d0d-istiod-ca-cert\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.568665 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.568452    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-certs\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.568665 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.568486    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-token\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.568665 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.568514    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhfjw\" (UniqueName: \"kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-kube-api-access-fhfjw\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.568665 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.568626    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.568865 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.568666    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-credential-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.568865 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.568693    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-data\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.568865 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.568853    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-podinfo\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.569018 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.568970    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-certs\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.569078 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.569029    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.569078 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.569053    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-data\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.569397 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.569297    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/576c2b0b-6909-4f31-99b5-1dd353650d0d-istiod-ca-cert\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.569518 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.569463    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-credential-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.570976 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.570953    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-envoy\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.571785 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.571764    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-podinfo\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.597917 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.597892    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-token\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.598037 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.598017    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhfjw\" (UniqueName: \"kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-kube-api-access-fhfjw\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.618391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.618329    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:05.749674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.749643    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"]
Oct 30 05:48:05.750506 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:48:05.750479    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode09beacd_ed50_4e0d_b633_8900fae01778.slice/crio-9f18d6fe44a8c243485be7dadda02d90054b90492555f1f7301caa4f9f595c95 WatchSource:0}: Error finding container 9f18d6fe44a8c243485be7dadda02d90054b90492555f1f7301caa4f9f595c95: Status 404 returned error can't find the container with id 9f18d6fe44a8c243485be7dadda02d90054b90492555f1f7301caa4f9f595c95
Oct 30 05:48:05.752955 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.752926    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:48:05.753024 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.752989    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:48:05.753024 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.753017    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:48:05.928427 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.928397    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"]
Oct 30 05:48:05.929417 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:48:05.929393    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod576c2b0b_6909_4f31_99b5_1dd353650d0d.slice/crio-17459420a58f592ebe17ba6aeaeb3909f5b841b83b246ee25de6a846e0791879 WatchSource:0}: Error finding container 17459420a58f592ebe17ba6aeaeb3909f5b841b83b246ee25de6a846e0791879: Status 404 returned error can't find the container with id 17459420a58f592ebe17ba6aeaeb3909f5b841b83b246ee25de6a846e0791879
Oct 30 05:48:05.932486 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.932454    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:48:05.932647 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.932535    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:48:05.932647 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:05.932573    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:48:06.560403 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:06.560360    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" event={"ID":"576c2b0b-6909-4f31-99b5-1dd353650d0d","Type":"ContainerStarted","Data":"e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1"}
Oct 30 05:48:06.560892 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:06.560426    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" event={"ID":"576c2b0b-6909-4f31-99b5-1dd353650d0d","Type":"ContainerStarted","Data":"17459420a58f592ebe17ba6aeaeb3909f5b841b83b246ee25de6a846e0791879"}
Oct 30 05:48:06.562300 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:06.562269    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" event={"ID":"e09beacd-ed50-4e0d-b633-8900fae01778","Type":"ContainerStarted","Data":"a1d93c8389727aaac91a9c6f6f0b51519318de5f913bc3239b7402a093d27468"}
Oct 30 05:48:06.562403 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:06.562332    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" event={"ID":"e09beacd-ed50-4e0d-b633-8900fae01778","Type":"ContainerStarted","Data":"9f18d6fe44a8c243485be7dadda02d90054b90492555f1f7301caa4f9f595c95"}
Oct 30 05:48:06.617610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:06.615129    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" podStartSLOduration=1.615107968 podStartE2EDuration="1.615107968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:48:06.612879732 +0000 UTC m=+990.358165065" watchObservedRunningTime="2026-10-30 05:48:06.615107968 +0000 UTC m=+990.360393300"
Oct 30 05:48:06.620294 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:06.620265    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:06.876713 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:06.873740    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:07.422295 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:07.422260    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:07.428614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:07.427821    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 05:48:07.428614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:07.427881    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 05:48:07.605425 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:07.605381    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:07.621494 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:07.621459    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 05:48:07.621678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:07.621514    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 05:48:08.422148 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:08.422116    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 05:48:08.422318 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:08.422161    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 05:48:08.619068 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:08.619022    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 05:48:08.619435 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:08.619087    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 05:48:08.806663 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:08.806506    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:09.422299 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:09.422264    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 05:48:09.422498 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:09.422327    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 05:48:09.541614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:09.534721    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:09.618768 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:09.618733    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 05:48:09.618937 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:09.618785    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 05:48:10.422259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:10.422227    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 05:48:10.422721 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:10.422281    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 05:48:10.619338 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:10.619300    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 05:48:10.619535 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:10.619356    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 05:48:11.422469 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:11.422435    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 05:48:11.423070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:11.422482    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 05:48:11.505855 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:11.505810    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:11.618998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:11.618964    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 05:48:11.619196 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:11.619068    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 05:48:12.422272 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:12.422232    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 05:48:12.422531 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:12.422306    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 05:48:12.619069 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:12.619040    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 05:48:12.619240 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:12.619113    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 05:48:13.144105 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:13.144070    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:13.426788 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:13.426755    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:13.427245 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:13.427204    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:13.428054 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:13.427925    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 05:48:13.493960 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:13.493903    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" podStartSLOduration=8.493885506 podStartE2EDuration="8.493885506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:48:06.675516957 +0000 UTC m=+990.420802289" watchObservedRunningTime="2026-10-30 05:48:13.493885506 +0000 UTC m=+997.239170839"
Oct 30 05:48:13.623638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:13.623610    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:13.623898 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:13.623879    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:13.624683 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:13.624662    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 05:48:14.667512 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:14.667478    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:16.439717 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:16.439681    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:18.121658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:18.121625    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:19.868853 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:19.868824    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:20.842561 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.842522    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"]
Oct 30 05:48:20.847783 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.847760    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:20.865433 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.865410    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-a35835-openshift-default-dockercfg-rbcbq\""
Oct 30 05:48:20.883753 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.883724    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"]
Oct 30 05:48:20.924428 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.924393    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzns7\" (UniqueName: \"kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-kube-api-access-lzns7\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:20.924428 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.924429    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77d0685c-e08d-4042-b2da-03139934718a-istio-podinfo\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:20.924661 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.924464    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:20.924661 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.924549    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77d0685c-e08d-4042-b2da-03139934718a-istiod-ca-cert\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:20.924661 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.924616    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-data\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:20.924825 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.924765    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-credential-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:20.924825 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.924799    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-certs\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:20.924954 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.924940    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-istio-token\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:20.925012 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:20.925000    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-envoy\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.026515 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.026479    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.026515 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.026513    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77d0685c-e08d-4042-b2da-03139934718a-istiod-ca-cert\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.026802 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.026534    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-data\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.026802 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.026569    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-credential-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.026802 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.026583    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-certs\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.026802 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.026693    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-istio-token\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.026802 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.026748    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-envoy\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.027075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.026839    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzns7\" (UniqueName: \"kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-kube-api-access-lzns7\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.027075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.026870    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77d0685c-e08d-4042-b2da-03139934718a-istio-podinfo\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.027075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.026989    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.027075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.027052    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-data\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.027288 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.027151    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-certs\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.027288 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.027258    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-credential-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.027400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.027380    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77d0685c-e08d-4042-b2da-03139934718a-istiod-ca-cert\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.029324 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.029300    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-envoy\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.029617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.029581    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77d0685c-e08d-4042-b2da-03139934718a-istio-podinfo\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.053266 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.053238    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzns7\" (UniqueName: \"kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-kube-api-access-lzns7\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.054453 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.054428    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-istio-token\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.146940 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.146856    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"]
Oct 30 05:48:21.151053 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.151032    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.162517 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.162492    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:21.167955 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.167930    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-128436-openshift-default-dockercfg-dwwj9\""
Oct 30 05:48:21.185633 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.181802    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"]
Oct 30 05:48:21.228979 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.228930    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-token\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.229152 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.229008    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istiod-ca-cert\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.229240 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.229156    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-envoy\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.229240 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.229198    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-certs\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.229357 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.229242    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-data\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.229357 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.229275    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.229357 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.229302    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-credential-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.229357 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.229332    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkwzd\" (UniqueName: \"kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-kube-api-access-wkwzd\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.229568 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.229407    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-podinfo\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.331197 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.331159    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-token\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.331391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.331226    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istiod-ca-cert\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.331391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.331317    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-envoy\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.331391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.331352    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-certs\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.331576 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.331390    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-data\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.331576 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.331422    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.331576 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.331450    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-credential-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.331576 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.331477    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkwzd\" (UniqueName: \"kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-kube-api-access-wkwzd\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.331576 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.331543    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-podinfo\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.331850 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.331815    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-certs\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.331910 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.331883    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.331972 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.331936    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-data\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.332134 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.332101    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-credential-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.332254 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.332137    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istiod-ca-cert\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.333957 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.333932    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-envoy\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.334161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.334143    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-podinfo\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.356857 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.356830    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-token\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.361316 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.361298    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkwzd\" (UniqueName: \"kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-kube-api-access-wkwzd\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.466313 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.466277    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:21.513866 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.513833    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"]
Oct 30 05:48:21.517811 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:48:21.517781    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d0685c_e08d_4042_b2da_03139934718a.slice/crio-9a64a975a22449c736dde4c5cd9fcd5f909bad9784c638701513b207b6bb648d WatchSource:0}: Error finding container 9a64a975a22449c736dde4c5cd9fcd5f909bad9784c638701513b207b6bb648d: Status 404 returned error can't find the container with id 9a64a975a22449c736dde4c5cd9fcd5f909bad9784c638701513b207b6bb648d
Oct 30 05:48:21.524058 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.523830    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:48:21.524058 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.523908    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:48:21.524058 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.523945    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:48:21.612095 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.612060    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:21.674459 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.674423    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" event={"ID":"77d0685c-e08d-4042-b2da-03139934718a","Type":"ContainerStarted","Data":"9d009bccc23a4ec402993f4193c30b55d9ae8327c27935fbb640e515e5d6a5f1"}
Oct 30 05:48:21.674667 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.674484    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" event={"ID":"77d0685c-e08d-4042-b2da-03139934718a","Type":"ContainerStarted","Data":"9a64a975a22449c736dde4c5cd9fcd5f909bad9784c638701513b207b6bb648d"}
Oct 30 05:48:21.728580 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.728467    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" podStartSLOduration=1.7284479209999999 podStartE2EDuration="1.728447921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:48:21.723933251 +0000 UTC m=+1005.469218583" watchObservedRunningTime="2026-10-30 05:48:21.728447921 +0000 UTC m=+1005.473733255"
Oct 30 05:48:21.784397 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:48:21.784359    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2c8f49_dfe8_4b28_9634_c695bc74bf18.slice/crio-7cde63dd77ca80bd9f11c07598da2fb70664a4be6db1c376a15a256fa7c0ec2d WatchSource:0}: Error finding container 7cde63dd77ca80bd9f11c07598da2fb70664a4be6db1c376a15a256fa7c0ec2d: Status 404 returned error can't find the container with id 7cde63dd77ca80bd9f11c07598da2fb70664a4be6db1c376a15a256fa7c0ec2d
Oct 30 05:48:21.785781 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.785754    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"]
Oct 30 05:48:21.787117 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.787078    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:48:21.787189 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.787171    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:48:21.787237 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:21.787222    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:48:22.163178 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:22.163141    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:22.165049 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:22.165024    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 05:48:22.165173 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:22.165075    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 05:48:22.689110 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:22.689060    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" event={"ID":"cf2c8f49-dfe8-4b28-9634-c695bc74bf18","Type":"ContainerStarted","Data":"3706b3da831df468f2cf7baa120dcee51fd012b0968d31ed9c5eab8958265120"}
Oct 30 05:48:22.689298 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:22.689122    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" event={"ID":"cf2c8f49-dfe8-4b28-9634-c695bc74bf18","Type":"ContainerStarted","Data":"7cde63dd77ca80bd9f11c07598da2fb70664a4be6db1c376a15a256fa7c0ec2d"}
Oct 30 05:48:22.755643 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:22.755562    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" podStartSLOduration=1.755543207 podStartE2EDuration="1.755543207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:48:22.754833842 +0000 UTC m=+1006.500119213" watchObservedRunningTime="2026-10-30 05:48:22.755543207 +0000 UTC m=+1006.500828544"
Oct 30 05:48:23.174612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:23.164033    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 05:48:23.174612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:23.164092    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 05:48:23.468846 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:23.468753    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:23.469222 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:23.469197    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 05:48:23.469393 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:23.469371    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 05:48:23.615846 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:23.615819    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:24.163617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:24.163126    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 05:48:24.163617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:24.163208    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 05:48:24.467670 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:24.467031    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 05:48:24.467670 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:24.467094    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 05:48:25.164616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:25.162988    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 05:48:25.164616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:25.163050    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 05:48:25.451665 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:25.451101    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:25.468464 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:25.468011    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 05:48:25.468464 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:25.468075    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 05:48:26.163004 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:26.162968    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 05:48:26.163303 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:26.163278    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 05:48:26.470672 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:26.466991    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 05:48:26.470672 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:26.467056    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 05:48:27.169335 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:27.169297    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 05:48:27.169665 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:27.169636    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 05:48:27.468213 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:27.468111    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 05:48:27.468499 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:27.468451    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 05:48:27.493306 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:27.493264    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:27.660211 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:27.660177    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:28.163001 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:28.162959    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 05:48:28.163187 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:28.163033    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 05:48:28.467519 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:28.467429    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 05:48:28.467710 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:28.467533    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 05:48:28.631078 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:28.630995    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:29.167528 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:29.167502    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:29.167796 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:29.167779    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:29.168622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:29.168572    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 05:48:29.394805 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:29.394772    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:29.471445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:29.471375    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:29.471655 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:29.471635    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:29.472359 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:29.472341    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 05:48:31.243284 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:31.243248    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:32.554606 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:48:32.554556    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:48:32.908875 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:32.908842    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:34.556012 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.555977    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-7fg5s-j2hhc"]
Oct 30 05:48:34.560295 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.560273    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.592836 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.592797    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lv7bs\""
Oct 30 05:48:34.613988 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.613926    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-7fg5s-j2hhc"]
Oct 30 05:48:34.631062 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.631035    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:34.633491 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.633469    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szsvc\" (UniqueName: \"kubernetes.io/projected/ba47e793-a31e-428e-a4ef-4eebfd5956d1-kube-api-access-szsvc\") pod \"periodic-gathering-7fg5s-j2hhc\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") " pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.633620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.633501    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-service-ca-bundle\") pod \"periodic-gathering-7fg5s-j2hhc\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") " pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.633620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.633529    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ba47e793-a31e-428e-a4ef-4eebfd5956d1-archives-path\") pod \"periodic-gathering-7fg5s-j2hhc\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") " pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.633700 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.633672    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-trusted-ca-bundle\") pod \"periodic-gathering-7fg5s-j2hhc\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") " pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.734527 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.734497    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szsvc\" (UniqueName: \"kubernetes.io/projected/ba47e793-a31e-428e-a4ef-4eebfd5956d1-kube-api-access-szsvc\") pod \"periodic-gathering-7fg5s-j2hhc\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") " pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.734527 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.734529    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-service-ca-bundle\") pod \"periodic-gathering-7fg5s-j2hhc\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") " pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.734800 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.734558    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ba47e793-a31e-428e-a4ef-4eebfd5956d1-archives-path\") pod \"periodic-gathering-7fg5s-j2hhc\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") " pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.734800 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.734651    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-trusted-ca-bundle\") pod \"periodic-gathering-7fg5s-j2hhc\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") " pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.735157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.735131    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ba47e793-a31e-428e-a4ef-4eebfd5956d1-archives-path\") pod \"periodic-gathering-7fg5s-j2hhc\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") " pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.735286 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.735261    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-service-ca-bundle\") pod \"periodic-gathering-7fg5s-j2hhc\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") " pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.735623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.735570    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-trusted-ca-bundle\") pod \"periodic-gathering-7fg5s-j2hhc\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") " pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.764322 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.764290    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szsvc\" (UniqueName: \"kubernetes.io/projected/ba47e793-a31e-428e-a4ef-4eebfd5956d1-kube-api-access-szsvc\") pod \"periodic-gathering-7fg5s-j2hhc\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") " pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:34.874319 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:34.874237    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:48:35.198487 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:35.198459    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-7fg5s-j2hhc"]
Oct 30 05:48:35.198890 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:48:35.198868    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba47e793_a31e_428e_a4ef_4eebfd5956d1.slice/crio-dc59426f31ba998a6324185ad7d29a1e7194efa2c71ab95915677b68efc60479 WatchSource:0}: Error finding container dc59426f31ba998a6324185ad7d29a1e7194efa2c71ab95915677b68efc60479: Status 404 returned error can't find the container with id dc59426f31ba998a6324185ad7d29a1e7194efa2c71ab95915677b68efc60479
Oct 30 05:48:35.778650 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:35.778604    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-7fg5s-j2hhc" event={"ID":"ba47e793-a31e-428e-a4ef-4eebfd5956d1","Type":"ContainerStarted","Data":"e3abf893829cde7005e59197fd8d9667928cede929377e357b6f06922793d62e"}
Oct 30 05:48:35.779164 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:35.778700    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-7fg5s-j2hhc" event={"ID":"ba47e793-a31e-428e-a4ef-4eebfd5956d1","Type":"ContainerStarted","Data":"dc59426f31ba998a6324185ad7d29a1e7194efa2c71ab95915677b68efc60479"}
Oct 30 05:48:35.827157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:35.827097    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-7fg5s-j2hhc" podStartSLOduration=1.8270814290000001 podStartE2EDuration="1.827081429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:48:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:48:35.82501826 +0000 UTC m=+1019.570303593" watchObservedRunningTime="2026-10-30 05:48:35.827081429 +0000 UTC m=+1019.572366781"
Oct 30 05:48:36.283226 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:36.283185    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:36.571061 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:36.570958    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:38.203062 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:38.203026    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:38.374104 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:38.374072    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:39.911955 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:39.911919    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:41.510238 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:41.510205    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:43.071757 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:43.071581    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:44.706867 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:44.706836    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:46.201137 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:46.201095    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:47.937627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:47.937165    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:49.623740 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:49.623701    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:51.116153 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:51.116122    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:52.470889 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:52.470852    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:52.982321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:52.982284    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:54.540755 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:54.540712    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:56.201302 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:56.201269    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:57.761867 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:57.761831    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:48:59.418658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:48:59.418614    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:01.022939 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:01.022899    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:02.522835 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:02.522803    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:02.564270 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:49:02.564229    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:49:04.129278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:04.129241    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:05.268293 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.268260    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"]
Oct 30 05:49:05.274171 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.274146    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.313263 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.313218    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"]
Oct 30 05:49:05.331292 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.331266    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-data\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.331886 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.331858    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-podinfo\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.332172 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.332149    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98tth\" (UniqueName: \"kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-kube-api-access-98tth\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.332277 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.332250    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-envoy\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.332343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.332275    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-token\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.332343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.332335    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istiod-ca-cert\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.332446 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.332364    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-credential-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.332446 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.332438    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.332548 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.332472    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-certs\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.439497 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.439370    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-envoy\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.439786 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.439763    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-token\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.439936 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.439921    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istiod-ca-cert\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.440032 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.440019    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-credential-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.440149 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.440137    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.440245 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.440232    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-certs\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.440429 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.440416    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-data\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.440525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.440514    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-podinfo\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.440676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.440661    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98tth\" (UniqueName: \"kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-kube-api-access-98tth\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.441868 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.441845    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.445897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.442934    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-data\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.445897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.443228    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-certs\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.446076 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.445960    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-credential-socket\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.453291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.453267    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istiod-ca-cert\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.454084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.454058    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-envoy\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.463239 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.463205    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-podinfo\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.466655 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.466633    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98tth\" (UniqueName: \"kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-kube-api-access-98tth\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.471072 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.471048    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-token\") pod \"e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") " pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.590073 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.589986    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:05.935795 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.935759    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:05.952684 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.952650    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"]
Oct 30 05:49:05.959780 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:49:05.959727    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf5bf1c5_1f01_4cb6_9cb6_21a4516f8877.slice/crio-6d8d6066c275ed7e31d03e853db3799f5b76c74214512e5bd75cd81a8515eaa3 WatchSource:0}: Error finding container 6d8d6066c275ed7e31d03e853db3799f5b76c74214512e5bd75cd81a8515eaa3: Status 404 returned error can't find the container with id 6d8d6066c275ed7e31d03e853db3799f5b76c74214512e5bd75cd81a8515eaa3
Oct 30 05:49:05.962900 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.962376    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:05.962900 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.962457    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:05.962900 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.962498    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:05.966918 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:05.966881    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" event={"ID":"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877","Type":"ContainerStarted","Data":"6d8d6066c275ed7e31d03e853db3799f5b76c74214512e5bd75cd81a8515eaa3"}
Oct 30 05:49:06.284376 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.284278    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"]
Oct 30 05:49:06.289454 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.289426    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.349448 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.349364    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"]
Oct 30 05:49:06.356854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.356330    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istiod-ca-cert\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.356854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.356402    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-token\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.356854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.356440    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-data\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.356854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.356486    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-podinfo\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.356854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.356583    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-envoy\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.356854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.356707    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-certs\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.356854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.356743    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-credential-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.356854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.356773    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tjqc\" (UniqueName: \"kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-kube-api-access-6tjqc\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.357719 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.357474    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.459304 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.459267    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-podinfo\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.459458 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.459408    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-envoy\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.459533 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.459514    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-certs\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.459607 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.459551    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-credential-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.459607 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.459582    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tjqc\" (UniqueName: \"kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-kube-api-access-6tjqc\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.459735 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.459695    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.459790 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.459762    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istiod-ca-cert\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.459844 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.459831    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-token\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.459900 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.459872    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-data\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.460136 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.460034    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.460269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.460249    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-certs\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.460512 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.460492    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-credential-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.460779 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.460755    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istiod-ca-cert\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.461349 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.461151    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-data\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.463174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.463153    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-envoy\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.463513 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.463491    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-podinfo\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.488198 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.488157    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-token\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.488415 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.488391    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tjqc\" (UniqueName: \"kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-kube-api-access-6tjqc\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.613013 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.611848    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:06.854067 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.854034    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:06.987615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:06.970897    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"]
Oct 30 05:49:07.006832 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:07.004542    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:07.006832 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:07.004655    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:07.006832 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:07.004698    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:07.045351 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:07.044879    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" event={"ID":"b718d3d8-55d8-45cc-82e2-553b3e2e1163","Type":"ContainerStarted","Data":"557bc42a1ee765d64e49151eedfaa4d57d96a6539cb743fdddc3dd9d306ad2cd"}
Oct 30 05:49:07.054473 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:07.054436    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" event={"ID":"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877","Type":"ContainerStarted","Data":"c6f4f8cb9e4e359b620c1e7cce75f393ee594fc2cda1eff3e225ba642035bc30"}
Oct 30 05:49:07.112912 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:07.112859    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" podStartSLOduration=2.112842768 podStartE2EDuration="2.112842768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:49:07.111934037 +0000 UTC m=+1050.857219369" watchObservedRunningTime="2026-10-30 05:49:07.112842768 +0000 UTC m=+1050.858128099"
Oct 30 05:49:07.590258 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:07.590229    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:07.591649 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:07.591628    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 05:49:07.591709 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:07.591686    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 05:49:07.849675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:07.841570    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:08.082622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:08.082265    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" event={"ID":"b718d3d8-55d8-45cc-82e2-553b3e2e1163","Type":"ContainerStarted","Data":"91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5"}
Oct 30 05:49:08.182119 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:08.179177    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" podStartSLOduration=2.179157868 podStartE2EDuration="2.179157868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:49:08.157087282 +0000 UTC m=+1051.902372619" watchObservedRunningTime="2026-10-30 05:49:08.179157868 +0000 UTC m=+1051.924443201"
Oct 30 05:49:08.604671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:08.604006    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 05:49:08.604671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:08.604070    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 05:49:08.618363 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:08.618332    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:08.627716 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:08.627686    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 05:49:08.640908 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:08.640867    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 05:49:09.600616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:09.591777    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:49:09.613627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:09.611625    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 05:49:09.613627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:09.611682    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 05:49:09.617610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:09.615967    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 05:49:09.617610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:09.616019    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 05:49:09.665625 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:09.662521    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:49:09.729303 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:09.729235    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:49:09.763617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:09.756747    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:49:09.791639 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:09.785300    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:49:09.861804 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:09.861689    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:49:09.975533 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:09.975486    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:49:10.039310 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:10.039263    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:49:10.108741 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:10.108695    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:49:10.592187 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:10.592152    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 05:49:10.592492 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:10.592462    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 05:49:10.617554 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:10.617522    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 05:49:10.618145 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:10.618111    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 05:49:10.760860 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:10.758678    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:49:10.760860 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:10.759371    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:49:10.830622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:10.826484    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:49:10.830622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:10.827430    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:49:10.916718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:10.887213    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 05:49:10.916718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:10.888233    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:49:11.129127 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:11.129075    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 05:49:11.137620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:11.129930    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:49:11.196677 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:11.195815    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": context deadline exceeded"
Oct 30 05:49:11.503662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:11.498914    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:49:11.596702 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:11.596225    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 05:49:11.596702 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:11.596399    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 05:49:11.611988 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:11.611958    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:49:11.623572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:11.614168    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 05:49:11.623572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:11.614219    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 05:49:11.623572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:11.622887    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:49:11.691358 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:11.689898    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:49:11.810170 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:11.810083    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:12.590767 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:12.590723    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 05:49:12.590948 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:12.590798    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 05:49:12.612930 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:12.612893    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 05:49:12.613089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:12.612993    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 05:49:13.590935 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:13.590899    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 05:49:13.591323 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:13.590968    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 05:49:13.612491 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:13.612456    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 05:49:13.612699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:13.612551    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 05:49:13.906523 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:13.906476    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:14.612553 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:14.612504    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 05:49:14.613092 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:14.612602    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 05:49:15.013303 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:15.013272    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:15.014128 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:15.014104    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:15.014886 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:15.014864    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:49:15.521547 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:15.521496    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:15.672187 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:15.671960    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:15.672697 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:15.672494    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:15.674137 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:15.674112    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 05:49:17.276662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:17.276627    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:18.727929 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:18.727896    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:20.286718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:20.286683    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:21.918253 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:21.918214    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:23.715738 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:23.715691    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:25.284795 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:25.284764    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:27.357881 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:27.357851    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:27.722392 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:27.722349    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:29.108954 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:29.108922    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:29.899411 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:29.899339    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:30.722409 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:30.722367    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:32.316961 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:32.316901    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:32.552186 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:49:32.552148    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:49:33.854176 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:33.854135    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:35.407710 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:35.407672    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:36.042010 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.041976    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"]
Oct 30 05:49:36.046618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.046566    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.073548 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.073522    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"]
Oct 30 05:49:36.193176 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.193139    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjd8c\" (UniqueName: \"kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-kube-api-access-gjd8c\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.193365 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.193196    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-envoy\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.193365 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.193294    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.193365 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.193359    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-istio-token\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.193484 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.193458    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-certs\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.193521 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.193495    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f92a6f0-defd-4770-b587-6829676b2572-istio-podinfo\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.193562 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.193521    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-credential-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.193632 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.193624    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-data\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.193673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.193648    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f92a6f0-defd-4770-b587-6829676b2572-istiod-ca-cert\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.294598 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.294498    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-certs\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.294598 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.294543    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f92a6f0-defd-4770-b587-6829676b2572-istio-podinfo\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.294598 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.294561    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-credential-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.294897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.294622    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-data\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.294897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.294646    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f92a6f0-defd-4770-b587-6829676b2572-istiod-ca-cert\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.294897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.294697    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjd8c\" (UniqueName: \"kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-kube-api-access-gjd8c\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.294897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.294754    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-envoy\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.294897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.294794    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.294897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.294875    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-istio-token\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.295196 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.294991    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-certs\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.295196 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.295023    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-credential-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.295398 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.295373    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-data\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.295494 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.295454    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f92a6f0-defd-4770-b587-6829676b2572-istiod-ca-cert\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.295557 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.295537    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.297291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.297275    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-envoy\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.297469 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.297450    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f92a6f0-defd-4770-b587-6829676b2572-istio-podinfo\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.322208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.322182    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-istio-token\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.322468 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.322447    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjd8c\" (UniqueName: \"kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-kube-api-access-gjd8c\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.362240 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.362212    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:36.381127 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.381094    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"]
Oct 30 05:49:36.385476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.385455    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.407084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.406827    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-podinfo\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.407084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.406971    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4df6\" (UniqueName: \"kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-kube-api-access-j4df6\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.407084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.407011    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-envoy\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.407084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.407043    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.407084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.407065    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-data\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.407572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.407096    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-certs\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.407572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.407170    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-credential-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.407572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.407288    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d92910fe-e3b0-41f6-92f3-aee750b56067-istiod-ca-cert\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.407572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.407342    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-token\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.416264 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.416230    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"]
Oct 30 05:49:36.513443 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.513093    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4df6\" (UniqueName: \"kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-kube-api-access-j4df6\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.513443 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.513155    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-envoy\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.513443 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.513199    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.513443 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.513224    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-data\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.513443 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.513252    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-certs\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.513443 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.513318    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-credential-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.513443 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.513357    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d92910fe-e3b0-41f6-92f3-aee750b56067-istiod-ca-cert\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.513443 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.513400    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-token\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.514005 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.513514    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-podinfo\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.515309 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.514392    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.515309 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.514658    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-credential-socket\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.515309 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.514730    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-certs\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.515309 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.515253    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-data\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.515309 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.515257    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d92910fe-e3b0-41f6-92f3-aee750b56067-istiod-ca-cert\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.516864 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.516840    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-envoy\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.517321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.517302    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-podinfo\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.540284 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.540254    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4df6\" (UniqueName: \"kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-kube-api-access-j4df6\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.540438 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.540351    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-token\") pod \"e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") " pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.707863 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.707832    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:36.707863 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.707853    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"]
Oct 30 05:49:36.708907 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:49:36.708878    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f92a6f0_defd_4770_b587_6829676b2572.slice/crio-5a3ff0785ff99a8d7e95be516c4bf93b096ea9d1f77a1483fc9063dcf272d2e8 WatchSource:0}: Error finding container 5a3ff0785ff99a8d7e95be516c4bf93b096ea9d1f77a1483fc9063dcf272d2e8: Status 404 returned error can't find the container with id 5a3ff0785ff99a8d7e95be516c4bf93b096ea9d1f77a1483fc9063dcf272d2e8
Oct 30 05:49:36.711727 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.711680    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:36.711847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.711785    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:36.711847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:36.711832    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:37.046209 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:37.046186    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"]
Oct 30 05:49:37.047667 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:49:37.047638    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd92910fe_e3b0_41f6_92f3_aee750b56067.slice/crio-29c1ce74af424b40d72cb130b4cb93a282f4b846319eb0473b89133e0d0da245 WatchSource:0}: Error finding container 29c1ce74af424b40d72cb130b4cb93a282f4b846319eb0473b89133e0d0da245: Status 404 returned error can't find the container with id 29c1ce74af424b40d72cb130b4cb93a282f4b846319eb0473b89133e0d0da245
Oct 30 05:49:37.050466 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:37.050426    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:37.050572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:37.050518    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:37.050572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:37.050559    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:37.225323 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:37.224701    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:37.495069 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:37.494795    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" event={"ID":"5f92a6f0-defd-4770-b587-6829676b2572","Type":"ContainerStarted","Data":"aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c"}
Oct 30 05:49:37.495069 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:37.494863    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" event={"ID":"5f92a6f0-defd-4770-b587-6829676b2572","Type":"ContainerStarted","Data":"5a3ff0785ff99a8d7e95be516c4bf93b096ea9d1f77a1483fc9063dcf272d2e8"}
Oct 30 05:49:37.496812 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:37.496789    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" event={"ID":"d92910fe-e3b0-41f6-92f3-aee750b56067","Type":"ContainerStarted","Data":"6bbc1fffb8cca94145e1bee5e9a0158052c6abeb8542301d1f7744c4f57cb108"}
Oct 30 05:49:37.496941 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:37.496843    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" event={"ID":"d92910fe-e3b0-41f6-92f3-aee750b56067","Type":"ContainerStarted","Data":"29c1ce74af424b40d72cb130b4cb93a282f4b846319eb0473b89133e0d0da245"}
Oct 30 05:49:37.548614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:37.545860    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" podStartSLOduration=1.545839816 podStartE2EDuration="1.545839816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:49:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:49:37.543025134 +0000 UTC m=+1081.288310467" watchObservedRunningTime="2026-10-30 05:49:37.545839816 +0000 UTC m=+1081.291125154"
Oct 30 05:49:37.594510 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:37.594451    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" podStartSLOduration=1.594437943 podStartE2EDuration="1.594437943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:49:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:49:37.593767109 +0000 UTC m=+1081.339052438" watchObservedRunningTime="2026-10-30 05:49:37.594437943 +0000 UTC m=+1081.339723274"
Oct 30 05:49:37.710060 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:37.710024    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:38.369614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.368063    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:38.369614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.368216    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 05:49:38.369614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.368262    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 05:49:38.572698 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.572651    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"]
Oct 30 05:49:38.595619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.588180    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.595619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.589155    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"]
Oct 30 05:49:38.639725 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.627766    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c78066\"/\"distinct-w2-c78066-kserve-self-signed-certs\""
Oct 30 05:49:38.663564 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.662172    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:38.694617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.694000    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-tmp-dir\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.694617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.694202    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-home\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.694617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.694291    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4rwc\" (UniqueName: \"kubernetes.io/projected/17de89f6-37da-48fb-97cc-f982d94e83a7-kube-api-access-k4rwc\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.694617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.694398    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-dshm\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.710611 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.694464    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17de89f6-37da-48fb-97cc-f982d94e83a7-tls-certs\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.710611 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.709784    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-model-cache\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.713174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.713142    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 05:49:38.713425 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.713403    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 05:49:38.811208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.811166    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-home\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.824838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.824796    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4rwc\" (UniqueName: \"kubernetes.io/projected/17de89f6-37da-48fb-97cc-f982d94e83a7-kube-api-access-k4rwc\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.825488 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.825461    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-dshm\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.826125 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.826107    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17de89f6-37da-48fb-97cc-f982d94e83a7-tls-certs\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.826721 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.826702    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-model-cache\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.827267 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.827250    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-tmp-dir\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.827777 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.827108    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-model-cache\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.828118 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.811876    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-home\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.828241 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.827634    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-tmp-dir\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.843058 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.843014    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-dshm\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.871663 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.858617    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4rwc\" (UniqueName: \"kubernetes.io/projected/17de89f6-37da-48fb-97cc-f982d94e83a7-kube-api-access-k4rwc\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.873382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.873319    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17de89f6-37da-48fb-97cc-f982d94e83a7-tls-certs\") pod \"distinct-w2-c78066-kserve-5b47cd8657-z7d6p\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") " pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:38.939427 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:38.938184    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:39.227450 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:39.223909    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:39.374616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:39.372972    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 05:49:39.374616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:39.373031    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 05:49:39.478315 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:39.478208    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"]
Oct 30 05:49:39.488718 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:49:39.488687    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17de89f6_37da_48fb_97cc_f982d94e83a7.slice/crio-6fc8bc4f70e3c6c71df251ff220e96e8403c90649f840e43886a68ce8d971e16 WatchSource:0}: Error finding container 6fc8bc4f70e3c6c71df251ff220e96e8403c90649f840e43886a68ce8d971e16: Status 404 returned error can't find the container with id 6fc8bc4f70e3c6c71df251ff220e96e8403c90649f840e43886a68ce8d971e16
Oct 30 05:49:39.511563 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:39.511528    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" event={"ID":"17de89f6-37da-48fb-97cc-f982d94e83a7","Type":"ContainerStarted","Data":"6fc8bc4f70e3c6c71df251ff220e96e8403c90649f840e43886a68ce8d971e16"}
Oct 30 05:49:39.708880 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:39.708850    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 05:49:39.709267 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:39.708902    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 05:49:40.363617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:40.362990    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 05:49:40.363617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:40.363052    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 05:49:40.518734 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:40.518700    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" event={"ID":"17de89f6-37da-48fb-97cc-f982d94e83a7","Type":"ContainerStarted","Data":"e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5"}
Oct 30 05:49:40.569614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:40.568546    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podStartSLOduration=2.568525461 podStartE2EDuration="2.568525461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:49:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:49:40.567996056 +0000 UTC m=+1084.313281391" watchObservedRunningTime="2026-10-30 05:49:40.568525461 +0000 UTC m=+1084.313810794"
Oct 30 05:49:40.709051 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:40.708978    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 05:49:40.710300 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:40.709666    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 05:49:40.986742 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:40.985718    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:41.284136 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:41.284036    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:41.377504 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:41.376711    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 05:49:41.377504 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:41.376775    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 05:49:41.709120 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:41.709082    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 05:49:41.709562 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:41.709142    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 05:49:42.368153 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:42.368119    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 05:49:42.368399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:42.368371    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 05:49:42.709030 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:42.708996    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 05:49:42.709240 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:42.709067    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 05:49:43.176456 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:43.176423    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:43.362783 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:43.362745    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 05:49:43.362989 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:43.362797    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 05:49:43.708773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:43.708729    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 05:49:43.708964 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:43.708783    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 05:49:44.511824 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:44.511790    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:44.512252 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:44.512191    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:44.513000 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:44.512977    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 05:49:44.833131 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:44.833030    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:44.833509 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:44.833490    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:44.834444 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:44.834423    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 05:49:44.922915 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:44.922886    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:46.607305 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:46.607264    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:48.418133 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:48.418102    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:48.938671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:48.938643    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:48.967352 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:48.966536    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:49.636447 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:49.636416    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:49:50.227965 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.227932    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:50.753667 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.753634    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"]
Oct 30 05:49:50.759272 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.759238    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:50.776936 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.776908    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-c78066-openshift-default-dockercfg-g958x\""
Oct 30 05:49:50.779602 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.779558    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"]
Oct 30 05:49:50.905089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.905051    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:50.905269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.905096    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-token\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:50.905269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.905191    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-certs\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:50.905269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.905221    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-envoy\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:50.905269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.905254    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mmjk\" (UniqueName: \"kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-kube-api-access-9mmjk\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:50.905402 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.905341    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-podinfo\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:50.905402 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.905397    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-data\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:50.905474 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.905432    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istiod-ca-cert\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:50.905561 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:50.905541    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-credential-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.006992 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.006908    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.006992 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.006950    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-token\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.007208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.007070    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-certs\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.007208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.007102    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-envoy\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.007208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.007136    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mmjk\" (UniqueName: \"kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-kube-api-access-9mmjk\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.007208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.007160    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-podinfo\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.007414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.007267    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-data\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.007414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.007299    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.007414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.007332    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istiod-ca-cert\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.007561 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.007439    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-credential-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.007561 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.007450    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-certs\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.007696 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.007666    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-data\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.007824 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.007795    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-credential-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.008139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.007967    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istiod-ca-cert\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.010041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.010019    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-envoy\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.011174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.011149    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-podinfo\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.032708 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.032682    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-token\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.032875 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.032852    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mmjk\" (UniqueName: \"kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-kube-api-access-9mmjk\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.078758 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.078722    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:51.113460 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.113425    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"]
Oct 30 05:49:51.119795 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.119768    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.163645 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.163245    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"]
Oct 30 05:49:51.210978 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.210934    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn24b\" (UniqueName: \"kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-kube-api-access-hn24b\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.210978 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.210977    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-certs\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.211228 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.211094    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-envoy\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.211228 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.211139    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977c71a-0612-4f50-876e-720196d43303-istio-podinfo\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.211340 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.211270    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-credential-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.211340 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.211312    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.211455 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.211363    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-data\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.211455 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.211444    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977c71a-0612-4f50-876e-720196d43303-istiod-ca-cert\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.211547 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.211491    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-istio-token\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.313160 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313065    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn24b\" (UniqueName: \"kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-kube-api-access-hn24b\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.313160 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313109    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-certs\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.313160 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313151    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-envoy\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.313476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313173    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977c71a-0612-4f50-876e-720196d43303-istio-podinfo\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.313476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313232    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-credential-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.313476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313264    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.313476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313307    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-data\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.313476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313381    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977c71a-0612-4f50-876e-720196d43303-istiod-ca-cert\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.313476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313429    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-istio-token\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.314101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313662    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-certs\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.314101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313730    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-data\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.314101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313883    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-credential-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.314101 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.313925    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-socket\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.314322 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.314187    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977c71a-0612-4f50-876e-720196d43303-istiod-ca-cert\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.315678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.315655    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-envoy\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.315884 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.315867    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977c71a-0612-4f50-876e-720196d43303-istio-podinfo\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.335905 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.335873    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"]
Oct 30 05:49:51.340215 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.340199    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.345425 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.345400    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-istio-token\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.345762 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.345739    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn24b\" (UniqueName: \"kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-kube-api-access-hn24b\") pod \"e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") " pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.360300 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.360270    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-1354a5-openshift-default-dockercfg-8c2kz\""
Oct 30 05:49:51.364286 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.364261    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"]
Oct 30 05:49:51.414525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.414465    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.414525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.414521    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2964d72a-feee-4fcb-8b03-5048919222cb-istio-podinfo\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.414956 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.414651    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-credential-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.414956 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.414702    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-istio-token\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.414956 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.414819    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2964d72a-feee-4fcb-8b03-5048919222cb-istiod-ca-cert\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.415118 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.415001    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-data\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.415118 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.415041    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-certs\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.415118 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.415068    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4q2n\" (UniqueName: \"kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-kube-api-access-r4q2n\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.415118 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.415116    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-envoy\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.421999 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:49:51.421975    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf2d37a5_3086_4ac9_892f_06ab233f1e70.slice/crio-b61fd416d8c3215e0eb2a11a983c1a196fad2572008fee9cedaed0447563af45 WatchSource:0}: Error finding container b61fd416d8c3215e0eb2a11a983c1a196fad2572008fee9cedaed0447563af45: Status 404 returned error can't find the container with id b61fd416d8c3215e0eb2a11a983c1a196fad2572008fee9cedaed0447563af45
Oct 30 05:49:51.424269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.424245    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"]
Oct 30 05:49:51.424573 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.424544    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:51.424679 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.424659    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:51.424744 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.424714    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:51.437128 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.437096    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:51.516740 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.516530    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-credential-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.516740 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.516748    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-istio-token\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.517269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.516856    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2964d72a-feee-4fcb-8b03-5048919222cb-istiod-ca-cert\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.517269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.517085    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-data\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.517269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.517138    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-certs\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.517269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.517163    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4q2n\" (UniqueName: \"kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-kube-api-access-r4q2n\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.517269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.517217    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-envoy\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.517555 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.517279    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.517555 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.517324    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2964d72a-feee-4fcb-8b03-5048919222cb-istio-podinfo\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.518621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.518509    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-credential-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.519763 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.519683    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2964d72a-feee-4fcb-8b03-5048919222cb-istiod-ca-cert\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.521613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.521061    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2964d72a-feee-4fcb-8b03-5048919222cb-istio-podinfo\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.521613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.521277    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-data\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.521613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.521461    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-certs\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.522958 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.522250    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.530632 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.528868    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-envoy\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.556288 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.556256    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-istio-token\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.556990 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.556966    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4q2n\" (UniqueName: \"kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-kube-api-access-r4q2n\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.640490 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.640401    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" event={"ID":"bf2d37a5-3086-4ac9-892f-06ab233f1e70","Type":"ContainerStarted","Data":"8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672"}
Oct 30 05:49:51.640773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.640756    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" event={"ID":"bf2d37a5-3086-4ac9-892f-06ab233f1e70","Type":"ContainerStarted","Data":"b61fd416d8c3215e0eb2a11a983c1a196fad2572008fee9cedaed0447563af45"}
Oct 30 05:49:51.654493 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.654452    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:51.692658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.690530    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" podStartSLOduration=1.6905131359999999 podStartE2EDuration="1.690513136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:49:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:49:51.68972446 +0000 UTC m=+1095.435009792" watchObservedRunningTime="2026-10-30 05:49:51.690513136 +0000 UTC m=+1095.435798469"
Oct 30 05:49:51.817330 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.817300    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"]
Oct 30 05:49:51.818549 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:49:51.818525    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode977c71a_0612_4f50_876e_720196d43303.slice/crio-3dcf1947e83e66dfad00c837d265fce1d95bd3ca290a718b0d830e274ec8c334 WatchSource:0}: Error finding container 3dcf1947e83e66dfad00c837d265fce1d95bd3ca290a718b0d830e274ec8c334: Status 404 returned error can't find the container with id 3dcf1947e83e66dfad00c837d265fce1d95bd3ca290a718b0d830e274ec8c334
Oct 30 05:49:51.821280 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.821244    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:51.821386 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.821322    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:51.821386 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.821366    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:51.898415 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.898346    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"]
Oct 30 05:49:51.903172 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.903141    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:51.920454 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.920423    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"]
Oct 30 05:49:51.922113 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:51.922088    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-1354a5\"/\"distinct-w6-1354a5-kserve-self-signed-certs\""
Oct 30 05:49:52.006697 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.006293    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"]
Oct 30 05:49:52.008522 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.008113    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:52.011534 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:49:52.011498    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2964d72a_feee_4fcb_8b03_5048919222cb.slice/crio-9b2dab4ec59b025225ce87d6e5ca17da0aaa54346549727b94933082cd300c4c WatchSource:0}: Error finding container 9b2dab4ec59b025225ce87d6e5ca17da0aaa54346549727b94933082cd300c4c: Status 404 returned error can't find the container with id 9b2dab4ec59b025225ce87d6e5ca17da0aaa54346549727b94933082cd300c4c
Oct 30 05:49:52.015653 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.015620    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:52.015851 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.015832    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:52.015982 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.015966    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:49:52.036174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.036129    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-dshm\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.036324 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.036240    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-tmp-dir\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.036399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.036324    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61de80c4-337b-4223-9a80-92781d29ff3b-tls-certs\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.036399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.036352    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27gsc\" (UniqueName: \"kubernetes.io/projected/61de80c4-337b-4223-9a80-92781d29ff3b-kube-api-access-27gsc\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.036399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.036390    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-model-cache\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.036560 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.036440    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-home\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.079820 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.079795    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:52.081516 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.081481    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 05:49:52.081629 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.081533    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" podUID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 05:49:52.139169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.137665    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-dshm\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.139169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.137763    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-tmp-dir\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.139169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.137845    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61de80c4-337b-4223-9a80-92781d29ff3b-tls-certs\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.139169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.137876    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27gsc\" (UniqueName: \"kubernetes.io/projected/61de80c4-337b-4223-9a80-92781d29ff3b-kube-api-access-27gsc\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.139169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.137919    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-model-cache\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.139169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.137969    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-home\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.139169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.139083    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-tmp-dir\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.140901 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.140847    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-model-cache\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.149325 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.141580    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61de80c4-337b-4223-9a80-92781d29ff3b-tls-certs\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.149752 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.149712    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-home\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.167844 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.161501    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-dshm\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.179392 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.179301    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27gsc\" (UniqueName: \"kubernetes.io/projected/61de80c4-337b-4223-9a80-92781d29ff3b-kube-api-access-27gsc\") pod \"distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") " pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.185197 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.185024    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:52.233733 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.233441    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:52.671976 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.671259    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" event={"ID":"2964d72a-feee-4fcb-8b03-5048919222cb","Type":"ContainerStarted","Data":"6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c"}
Oct 30 05:49:52.671976 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.671318    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" event={"ID":"2964d72a-feee-4fcb-8b03-5048919222cb","Type":"ContainerStarted","Data":"9b2dab4ec59b025225ce87d6e5ca17da0aaa54346549727b94933082cd300c4c"}
Oct 30 05:49:52.676073 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.676041    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" event={"ID":"e977c71a-0612-4f50-876e-720196d43303","Type":"ContainerStarted","Data":"848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b"}
Oct 30 05:49:52.676234 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.676088    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" event={"ID":"e977c71a-0612-4f50-876e-720196d43303","Type":"ContainerStarted","Data":"3dcf1947e83e66dfad00c837d265fce1d95bd3ca290a718b0d830e274ec8c334"}
Oct 30 05:49:52.680053 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.680003    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"]
Oct 30 05:49:52.680393 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:49:52.680371    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61de80c4_337b_4223_9a80_92781d29ff3b.slice/crio-28c0ec6fd373bdd6e574ab3be07f97308c8172eec2f2e428b3121739df05e23d WatchSource:0}: Error finding container 28c0ec6fd373bdd6e574ab3be07f97308c8172eec2f2e428b3121739df05e23d: Status 404 returned error can't find the container with id 28c0ec6fd373bdd6e574ab3be07f97308c8172eec2f2e428b3121739df05e23d
Oct 30 05:49:52.754678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.726695    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" podStartSLOduration=1.726671798 podStartE2EDuration="1.726671798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:49:52.721651836 +0000 UTC m=+1096.466937169" watchObservedRunningTime="2026-10-30 05:49:52.726671798 +0000 UTC m=+1096.471957131"
Oct 30 05:49:52.775211 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:52.775156    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" podStartSLOduration=1.77513727 podStartE2EDuration="1.77513727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:49:52.773046176 +0000 UTC m=+1096.518331505" watchObservedRunningTime="2026-10-30 05:49:52.77513727 +0000 UTC m=+1096.520422603"
Oct 30 05:49:53.084403 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:53.084310    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 05:49:53.085007 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:53.084974    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" podUID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 05:49:53.442620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:53.442330    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:49:53.442620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:53.442465    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 05:49:53.442620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:53.442511    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" podUID="e977c71a-0612-4f50-876e-720196d43303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 05:49:53.663620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:53.662746    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:53.674614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:53.670719    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Oct 30 05:49:53.674614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:53.670779    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" podUID="2964d72a-feee-4fcb-8b03-5048919222cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Oct 30 05:49:53.715707 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:53.715464    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" event={"ID":"61de80c4-337b-4223-9a80-92781d29ff3b","Type":"ContainerStarted","Data":"adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c"}
Oct 30 05:49:53.715707 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:53.715519    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" event={"ID":"61de80c4-337b-4223-9a80-92781d29ff3b","Type":"ContainerStarted","Data":"28c0ec6fd373bdd6e574ab3be07f97308c8172eec2f2e428b3121739df05e23d"}
Oct 30 05:49:53.789625 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:53.788183    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podStartSLOduration=2.788161851 podStartE2EDuration="2.788161851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:49:53.785937945 +0000 UTC m=+1097.531223279" watchObservedRunningTime="2026-10-30 05:49:53.788161851 +0000 UTC m=+1097.533447187"
Oct 30 05:49:54.089144 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:54.089041    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 05:49:54.089766 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:54.089735    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" podUID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 05:49:54.144710 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:54.144677    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:54.235616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:54.234887    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:49:54.510658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:54.510578    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 05:49:54.511222 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:54.511188    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" podUID="e977c71a-0612-4f50-876e-720196d43303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 05:49:54.682650 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:54.675733    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Oct 30 05:49:54.682650 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:54.675801    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" podUID="2964d72a-feee-4fcb-8b03-5048919222cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Oct 30 05:49:55.080053 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:55.080014    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 05:49:55.081136 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:55.081093    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" podUID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 05:49:55.444037 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:55.443996    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 05:49:55.445430 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:55.445395    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" podUID="e977c71a-0612-4f50-876e-720196d43303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 05:49:55.657212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:55.657179    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Oct 30 05:49:55.657495 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:55.657466    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" podUID="2964d72a-feee-4fcb-8b03-5048919222cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Oct 30 05:49:56.083003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:56.082966    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 05:49:56.083313 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:56.083284    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" podUID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 05:49:56.205275 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:56.205227    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:56.442662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:56.437975    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 05:49:56.442662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:56.438035    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" podUID="e977c71a-0612-4f50-876e-720196d43303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 05:49:56.693752 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:56.685834    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Oct 30 05:49:56.693752 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:56.685898    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" podUID="2964d72a-feee-4fcb-8b03-5048919222cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Oct 30 05:49:57.081671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:57.080997    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 05:49:57.081671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:57.081056    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" podUID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 05:49:57.443615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:57.442011    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 05:49:57.443615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:57.442072    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" podUID="e977c71a-0612-4f50-876e-720196d43303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 05:49:57.665616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:57.656107    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Oct 30 05:49:57.665616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:57.656185    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" podUID="2964d72a-feee-4fcb-8b03-5048919222cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Oct 30 05:49:58.101971 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:58.101934    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 05:49:58.102487 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:58.101998    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" podUID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 05:49:58.381661 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:58.380688    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:49:58.439663 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:58.437628    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 05:49:58.439663 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:58.437709    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" podUID="e977c71a-0612-4f50-876e-720196d43303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 05:49:58.662694 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:58.658016    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Oct 30 05:49:58.662694 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:58.658074    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" podUID="2964d72a-feee-4fcb-8b03-5048919222cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Oct 30 05:49:59.464671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:59.464225    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 05:49:59.464671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:59.464282    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" podUID="e977c71a-0612-4f50-876e-720196d43303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 05:49:59.591746 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:59.591708    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:59.592531 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:59.592504    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:59.596256 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:59.596233    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 05:49:59.677017 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:59.676980    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:59.677669 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:59.677643    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:49:59.678736 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:49:59.678713    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:50:00.290283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:00.290239    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:00.442109 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:00.442080    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:50:00.442387 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:00.442362    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:50:00.443205 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:00.443187    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 05:50:01.785618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.785573    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"]
Oct 30 05:50:01.789848 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.789819    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.813326 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.813300    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"]
Oct 30 05:50:01.823699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.823663    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-197e9b\"/\"distinct-2-w0-197e9b-kserve-self-signed-certs\""
Oct 30 05:50:01.882997 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.882953    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:01.895356 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.895321    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-tmp-dir\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.895557 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.895406    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/133f5193-04ff-4d67-8515-8e5338675d6c-tls-certs\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.895557 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.895453    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-home\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.895756 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.895723    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-dshm\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.895898 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.895817    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-model-cache\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.895970 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.895893    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjxdh\" (UniqueName: \"kubernetes.io/projected/133f5193-04ff-4d67-8515-8e5338675d6c-kube-api-access-xjxdh\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.997537 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.997504    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/133f5193-04ff-4d67-8515-8e5338675d6c-tls-certs\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.997747 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.997560    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-home\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.997831 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.997793    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-dshm\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.997884 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.997863    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-model-cache\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.997940 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.997906    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjxdh\" (UniqueName: \"kubernetes.io/projected/133f5193-04ff-4d67-8515-8e5338675d6c-kube-api-access-xjxdh\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.998022 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.998003    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-home\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.998070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.998011    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-tmp-dir\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.998277 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.998256    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-tmp-dir\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:01.998330 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:01.998310    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-model-cache\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:02.001769 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:02.001740    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-dshm\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:02.002037 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:02.002011    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/133f5193-04ff-4d67-8515-8e5338675d6c-tls-certs\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:02.027428 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:02.027395    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjxdh\" (UniqueName: \"kubernetes.io/projected/133f5193-04ff-4d67-8515-8e5338675d6c-kube-api-access-xjxdh\") pod \"distinct-2-w0-197e9b-kserve-75f486cf55-cghpj\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") " pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:02.053099 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:02.053011    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:02.110635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:02.110602    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:02.235062 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:02.234960    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:50:02.256664 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:02.255355    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:50:02.463893 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:02.463869    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"]
Oct 30 05:50:02.464398 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:50:02.464373    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod133f5193_04ff_4d67_8515_8e5338675d6c.slice/crio-38202af789d69b9b9dfc7bd5c03ca945766a4bbda36aecdc5183d32230a98d71 WatchSource:0}: Error finding container 38202af789d69b9b9dfc7bd5c03ca945766a4bbda36aecdc5183d32230a98d71: Status 404 returned error can't find the container with id 38202af789d69b9b9dfc7bd5c03ca945766a4bbda36aecdc5183d32230a98d71
Oct 30 05:50:02.554240 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:50:02.554200    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:50:02.837216 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:02.837126    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" event={"ID":"133f5193-04ff-4d67-8515-8e5338675d6c","Type":"ContainerStarted","Data":"a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201"}
Oct 30 05:50:02.837216 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:02.837166    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" event={"ID":"133f5193-04ff-4d67-8515-8e5338675d6c","Type":"ContainerStarted","Data":"38202af789d69b9b9dfc7bd5c03ca945766a4bbda36aecdc5183d32230a98d71"}
Oct 30 05:50:02.848644 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:02.848619    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:50:02.882467 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:02.882419    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podStartSLOduration=1.882406019 podStartE2EDuration="1.882406019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:50:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:50:02.881657008 +0000 UTC m=+1106.626942341" watchObservedRunningTime="2026-10-30 05:50:02.882406019 +0000 UTC m=+1106.627691352"
Oct 30 05:50:03.111627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:03.111523    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:03.802494 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:03.802464    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:05.418537 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:05.418487    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:07.104851 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:07.104815    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:08.621938 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:08.621902    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:10.142141 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:10.142101    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:10.893999 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:10.893955    5066 generic.go:356] "Generic (PLEG): container finished" podID="ba47e793-a31e-428e-a4ef-4eebfd5956d1" containerID="e3abf893829cde7005e59197fd8d9667928cede929377e357b6f06922793d62e" exitCode=0
Oct 30 05:50:10.894166 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:10.894002    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-7fg5s-j2hhc" event={"ID":"ba47e793-a31e-428e-a4ef-4eebfd5956d1","Type":"ContainerDied","Data":"e3abf893829cde7005e59197fd8d9667928cede929377e357b6f06922793d62e"}
Oct 30 05:50:11.730186 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:11.730154    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:12.127100 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.127070    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:12.203899 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.203859    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:12.377615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.375747    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:50:12.647625 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.647514    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-service-ca-bundle\") pod \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") "
Oct 30 05:50:12.674814 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.674769    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba47e793-a31e-428e-a4ef-4eebfd5956d1-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ba47e793-a31e-428e-a4ef-4eebfd5956d1-archives-path\") pod \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") "
Oct 30 05:50:12.675195 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.675158    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba47e793-a31e-428e-a4ef-4eebfd5956d1-kube-api-access-szsvc\" (UniqueName: \"kubernetes.io/projected/ba47e793-a31e-428e-a4ef-4eebfd5956d1-kube-api-access-szsvc\") pod \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") "
Oct 30 05:50:12.675539 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.675516    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-trusted-ca-bundle\") pod \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\" (UID: \"ba47e793-a31e-428e-a4ef-4eebfd5956d1\") "
Oct 30 05:50:12.676356 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.673656    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-service-ca-bundle" pod "ba47e793-a31e-428e-a4ef-4eebfd5956d1" (UID: "ba47e793-a31e-428e-a4ef-4eebfd5956d1"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:50:12.676505 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.676332    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba47e793-a31e-428e-a4ef-4eebfd5956d1-archives-path" pod "ba47e793-a31e-428e-a4ef-4eebfd5956d1" (UID: "ba47e793-a31e-428e-a4ef-4eebfd5956d1"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:50:12.678693 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.678572    5066 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-service-ca-bundle\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:50:12.682205 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.682083    5066 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ba47e793-a31e-428e-a4ef-4eebfd5956d1-archives-path\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:50:12.682349 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.681923    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-trusted-ca-bundle" pod "ba47e793-a31e-428e-a4ef-4eebfd5956d1" (UID: "ba47e793-a31e-428e-a4ef-4eebfd5956d1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:50:12.726032 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.725995    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba47e793-a31e-428e-a4ef-4eebfd5956d1-kube-api-access-szsvc" pod "ba47e793-a31e-428e-a4ef-4eebfd5956d1" (UID: "ba47e793-a31e-428e-a4ef-4eebfd5956d1"). InnerVolumeSpecName "kube-api-access-szsvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:50:12.784084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.783990    5066 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba47e793-a31e-428e-a4ef-4eebfd5956d1-trusted-ca-bundle\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:50:12.784084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.784028    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szsvc\" (UniqueName: \"kubernetes.io/projected/ba47e793-a31e-428e-a4ef-4eebfd5956d1-kube-api-access-szsvc\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:50:12.969615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.966852    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-7fg5s-j2hhc" event={"ID":"ba47e793-a31e-428e-a4ef-4eebfd5956d1","Type":"ContainerDied","Data":"dc59426f31ba998a6324185ad7d29a1e7194efa2c71ab95915677b68efc60479"}
Oct 30 05:50:12.969615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.966953    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc59426f31ba998a6324185ad7d29a1e7194efa2c71ab95915677b68efc60479"
Oct 30 05:50:12.969615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:12.967072    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-7fg5s-j2hhc"
Oct 30 05:50:13.587943 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:13.587630    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:13.705928 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:13.705166    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:13.777340 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:13.777303    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:13.984346 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:13.984269    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:14.072659 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:14.070936    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:50:14.767797 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:14.767751    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:14.768473 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:14.768447    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:50:14.815650 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:14.814312    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:14.897654 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:14.875880    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:14.992614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:14.988961    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:15.081667 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:15.081086    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:15.096613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:15.096541    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:50:15.223038 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:15.223000    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:15.229819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:15.229515    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:15.282610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:15.244930    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:15.282945 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:15.245112    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:50:15.547562 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:15.546620    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:50:15.580613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:15.578707    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:15.581718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:15.581689    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:15.581918 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:15.581896    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:17.230773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:17.230680    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:18.993931 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:18.993897    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:20.588812 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.588782    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"]
Oct 30 05:50:20.589515 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.589488    5066 cpu_manager.go:414] "containerMap: removing container" podUID="ba47e793-a31e-428e-a4ef-4eebfd5956d1" containerName="insights-gathering"
Oct 30 05:50:20.589515 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.589513    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba47e793-a31e-428e-a4ef-4eebfd5956d1" containerName="insights-gathering"
Oct 30 05:50:20.589796 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.589707    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="ba47e793-a31e-428e-a4ef-4eebfd5956d1" containerName="insights-gathering"
Oct 30 05:50:20.594580 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.594561    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.618464 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.618427    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"]
Oct 30 05:50:20.621983 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.621541    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:20.624771 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.624727    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e2e3d0\"/\"distinct-2-w4-e2e3d0-kserve-self-signed-certs\""
Oct 30 05:50:20.686100 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.686062    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-home\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.686100 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.686104    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tls-certs\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.686346 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.686123    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-model-cache\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.686406 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.686384    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5xkg\" (UniqueName: \"kubernetes.io/projected/8bdc1785-b6ce-4de9-bda2-6127f1964b13-kube-api-access-j5xkg\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.686463 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.686447    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-dshm\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.686563 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.686542    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tmp-dir\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.784968 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.784914    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:20.789539 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.789495    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tls-certs\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.789757 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.789621    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-model-cache\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.789856 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.789835    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5xkg\" (UniqueName: \"kubernetes.io/projected/8bdc1785-b6ce-4de9-bda2-6127f1964b13-kube-api-access-j5xkg\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.790098 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.790072    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-dshm\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.790212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.790165    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tmp-dir\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.790285 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.790240    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-home\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.802614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.790743    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-home\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.802889 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.793826    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tls-certs\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.803003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.794061    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-model-cache\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.803076 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.797272    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-dshm\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.803148 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.797505    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tmp-dir\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.827672 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.820507    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5xkg\" (UniqueName: \"kubernetes.io/projected/8bdc1785-b6ce-4de9-bda2-6127f1964b13-kube-api-access-j5xkg\") pod \"distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") " pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:20.921671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:20.912463    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:21.082411 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.082172    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"]
Oct 30 05:50:21.088044 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.088020    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.107427 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.107399    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"]
Oct 30 05:50:21.121455 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.121411    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"]
Oct 30 05:50:21.130006 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.129979    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.152669 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.148562    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"]
Oct 30 05:50:21.196850 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.196781    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f65f330-0c39-4391-8de1-8d793b792708-istiod-ca-cert\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.196850 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.196817    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-istio-token\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.196850 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.196839    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.197086 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.196935    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-credential-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.197086 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.196960    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-podinfo\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.197086 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.196986    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f65f330-0c39-4391-8de1-8d793b792708-istio-podinfo\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.197086 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.197003    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snflv\" (UniqueName: \"kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-kube-api-access-snflv\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.197221 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.197121    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-envoy\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.197221 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.197170    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jtcb\" (UniqueName: \"kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-kube-api-access-8jtcb\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.197295 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.197269    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-credential-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.197381 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.197363    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-data\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.197445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.197403    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-token\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.197498 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.197488    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-certs\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.197570 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.197545    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istiod-ca-cert\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.197639 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.197579    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.197697 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.197642    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-certs\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.197697 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.197668    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-envoy\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.197805 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.197785    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-data\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.299230 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299195    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-istio-token\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.299412 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299240    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.299412 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299273    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-credential-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.299412 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299312    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-podinfo\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.299412 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299382    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f65f330-0c39-4391-8de1-8d793b792708-istio-podinfo\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.299412 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299412    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snflv\" (UniqueName: \"kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-kube-api-access-snflv\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.299708 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299491    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-envoy\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.299708 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299562    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jtcb\" (UniqueName: \"kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-kube-api-access-8jtcb\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.299708 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299690    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-credential-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.299861 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299728    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-data\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.299861 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299765    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-token\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.299861 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299788    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.299861 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299840    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-credential-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.300079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299870    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-certs\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.300079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299920    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istiod-ca-cert\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.300079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299965    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.300079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.299996    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-certs\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.300079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.300036    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-envoy\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.300343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.300084    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-data\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.300343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.300100    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-credential-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.300343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.300131    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f65f330-0c39-4391-8de1-8d793b792708-istiod-ca-cert\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.300343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.300184    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-data\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.300343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.300337    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-certs\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.300655 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.300494    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-data\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.300718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.300664    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istiod-ca-cert\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.300891 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.300716    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-certs\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.300891 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.300813    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-socket\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.301025 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.300997    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f65f330-0c39-4391-8de1-8d793b792708-istiod-ca-cert\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.302568 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.302547    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-envoy\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.302767 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.302749    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-envoy\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.302811 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.302762    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f65f330-0c39-4391-8de1-8d793b792708-istio-podinfo\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.302845 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.302833    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-podinfo\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.314230 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:50:21.314204    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bdc1785_b6ce_4de9_bda2_6127f1964b13.slice/crio-d1d3a1adc335f3c4bb84cf3bb2156a3c06e6327c8088604af1ea589f100c7ec3 WatchSource:0}: Error finding container d1d3a1adc335f3c4bb84cf3bb2156a3c06e6327c8088604af1ea589f100c7ec3: Status 404 returned error can't find the container with id d1d3a1adc335f3c4bb84cf3bb2156a3c06e6327c8088604af1ea589f100c7ec3
Oct 30 05:50:21.315761 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.315730    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"]
Oct 30 05:50:21.334733 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.334696    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-token\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.334897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.334874    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-istio-token\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.335333 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.335307    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jtcb\" (UniqueName: \"kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-kube-api-access-8jtcb\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.335494 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.335473    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snflv\" (UniqueName: \"kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-kube-api-access-snflv\") pod \"e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") " pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.403545 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.403511    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:21.453139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.453053    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:21.738251 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.738216    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"]
Oct 30 05:50:21.744538 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:50:21.744511    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f65f330_0c39_4391_8de1_8d793b792708.slice/crio-3975e12f8a6e63f7db04339546b6cc4657ac2cbb27dc95e5b9d2ca0a3d1de101 WatchSource:0}: Error finding container 3975e12f8a6e63f7db04339546b6cc4657ac2cbb27dc95e5b9d2ca0a3d1de101: Status 404 returned error can't find the container with id 3975e12f8a6e63f7db04339546b6cc4657ac2cbb27dc95e5b9d2ca0a3d1de101
Oct 30 05:50:21.747084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.747050    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:21.747172 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.747118    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:21.747172 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.747148    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:21.787328 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.787299    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"]
Oct 30 05:50:21.798996 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:50:21.798964    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39979b3b_2512_4d2f_b30f_6f59b843e2d6.slice/crio-f01765f15736faa66f14f43f90e26c736c8a6ab181d8825ccecf4b69900f395f WatchSource:0}: Error finding container f01765f15736faa66f14f43f90e26c736c8a6ab181d8825ccecf4b69900f395f: Status 404 returned error can't find the container with id f01765f15736faa66f14f43f90e26c736c8a6ab181d8825ccecf4b69900f395f
Oct 30 05:50:21.802444 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.802403    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:21.802563 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.802483    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:21.802563 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:21.802529    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:22.282046 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.282004    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" event={"ID":"8f65f330-0c39-4391-8de1-8d793b792708","Type":"ContainerStarted","Data":"7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5"}
Oct 30 05:50:22.282260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.282094    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" event={"ID":"8f65f330-0c39-4391-8de1-8d793b792708","Type":"ContainerStarted","Data":"3975e12f8a6e63f7db04339546b6cc4657ac2cbb27dc95e5b9d2ca0a3d1de101"}
Oct 30 05:50:22.285444 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.284572    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" event={"ID":"8bdc1785-b6ce-4de9-bda2-6127f1964b13","Type":"ContainerStarted","Data":"76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8"}
Oct 30 05:50:22.285673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.285648    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" event={"ID":"8bdc1785-b6ce-4de9-bda2-6127f1964b13","Type":"ContainerStarted","Data":"d1d3a1adc335f3c4bb84cf3bb2156a3c06e6327c8088604af1ea589f100c7ec3"}
Oct 30 05:50:22.287036 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.287011    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" event={"ID":"39979b3b-2512-4d2f-b30f-6f59b843e2d6","Type":"ContainerStarted","Data":"12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542"}
Oct 30 05:50:22.287156 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.287062    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" event={"ID":"39979b3b-2512-4d2f-b30f-6f59b843e2d6","Type":"ContainerStarted","Data":"f01765f15736faa66f14f43f90e26c736c8a6ab181d8825ccecf4b69900f395f"}
Oct 30 05:50:22.335178 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.335108    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" podStartSLOduration=1.3350849550000001 podStartE2EDuration="1.335084955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:50:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:50:22.335028411 +0000 UTC m=+1126.080313744" watchObservedRunningTime="2026-10-30 05:50:22.335084955 +0000 UTC m=+1126.080370289"
Oct 30 05:50:22.391028 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.390968    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" podStartSLOduration=1.39095301 podStartE2EDuration="1.39095301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:50:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:50:22.388024204 +0000 UTC m=+1126.133309536" watchObservedRunningTime="2026-10-30 05:50:22.39095301 +0000 UTC m=+1126.136238342"
Oct 30 05:50:22.406542 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.406502    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 05:50:22.406751 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.406569    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 05:50:22.410160 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.410132    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:22.450613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.450166    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podStartSLOduration=2.450145106 podStartE2EDuration="2.450145106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:50:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:50:22.448927418 +0000 UTC m=+1126.194212752" watchObservedRunningTime="2026-10-30 05:50:22.450145106 +0000 UTC m=+1126.195430439"
Oct 30 05:50:22.462873 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.462840    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:22.463284 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.463197    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 05:50:22.463467 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.463442    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 05:50:22.544671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.543994    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:22.913295 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:22.913265    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:23.404099 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:23.404063    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 05:50:23.404266 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:23.404134    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 05:50:23.459033 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:23.458994    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 05:50:23.459315 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:23.459283    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 05:50:24.252616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:24.252227    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:24.403904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:24.403874    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 05:50:24.404177 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:24.404155    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 05:50:24.464613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:24.462003    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 05:50:24.464613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:24.462079    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 05:50:25.403988 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:25.403955    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 05:50:25.404356 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:25.404017    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 05:50:25.454614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:25.453821    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 05:50:25.454614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:25.453879    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 05:50:26.304439 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:26.304328    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:26.408197 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:26.408159    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 05:50:26.408724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:26.408229    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 05:50:26.466615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:26.456969    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 05:50:26.466615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:26.457065    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 05:50:27.405081 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:27.405044    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 05:50:27.405266 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:27.405117    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 05:50:27.456003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:27.455963    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 05:50:27.456612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:27.456556    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 05:50:27.872362 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:27.872269    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:28.061358 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:28.061321    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:28.404896 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:28.404859    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 05:50:28.405090 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:28.404939    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 05:50:28.454542 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:28.454507    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Oct 30 05:50:28.454752 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:28.454572    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Oct 30 05:50:29.408892 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:29.408865    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:29.409330 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:29.409171    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:29.409947 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:29.409922    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 05:50:29.458216 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:29.458190    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:29.458509 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:29.458490    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:29.459248 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:29.459232    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 05:50:29.899061 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:29.898972    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:30.913275 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:30.913239    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:30.926227 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:30.926204    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:31.420623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:31.420563    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:31.487384 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:31.487355    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:32.554042 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:50:32.553997    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:50:33.096954 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:33.096917    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:34.656774 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:34.656633    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:35.701129 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.701089    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"]
Oct 30 05:50:35.706602 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.706569    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.717754 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.717718    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"]
Oct 30 05:50:35.721817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.721794    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.730542 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.730519    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"]
Oct 30 05:50:35.765809 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.765780    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"]
Oct 30 05:50:35.830476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.830436    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-envoy\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.830694 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.830500    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istiod-ca-cert\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.830694 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.830580    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-credential-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.830694 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.830630    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-envoy\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.830694 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.830654    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-credential-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.830882 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.830749    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-token\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.830882 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.830809    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vr5p\" (UniqueName: \"kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-kube-api-access-8vr5p\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.830882 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.830838    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-podinfo\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.830882 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.830858    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.831005 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.830960    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-data\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.831005 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.830989    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-data\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.831070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.831049    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqxs5\" (UniqueName: \"kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-kube-api-access-rqxs5\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.831103 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.831079    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istiod-ca-cert\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.831150 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.831137    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-certs\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.831206 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.831194    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-certs\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.831244 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.831216    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-podinfo\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.831274 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.831245    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.831364 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.831348    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-token\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.932969 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.932913    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-data\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.933144 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933027    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-data\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.933144 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933082    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqxs5\" (UniqueName: \"kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-kube-api-access-rqxs5\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.933144 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933137    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istiod-ca-cert\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.933268 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933197    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-certs\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.933268 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933238    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-certs\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.933376 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933266    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-podinfo\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.933376 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933314    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.933476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933371    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-token\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.933476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933405    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-data\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.933624 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933478    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-envoy\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.933624 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933601    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istiod-ca-cert\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.933739 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933638    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-credential-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.933739 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933646    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-certs\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.933739 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933668    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-envoy\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.933739 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933704    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-credential-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.933950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933752    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-token\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.933950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933824    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vr5p\" (UniqueName: \"kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-kube-api-access-8vr5p\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.933950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933862    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-podinfo\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.933950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933892    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.934139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.933976    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-credential-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.934139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.934077    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.934787 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.934237    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-certs\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.934787 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.934503    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-credential-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.934787 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.934515    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istiod-ca-cert\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.934787 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.934623    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-socket\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.934787 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.934751    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istiod-ca-cert\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.935104 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.934969    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-data\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.936539 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.936515    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-envoy\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.936947 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.936920    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-envoy\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.937386 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.937365    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-podinfo\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.937640 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.937624    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-podinfo\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.960476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.960408    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-token\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:35.960476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.960434    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-token\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.960693 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.960513    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vr5p\" (UniqueName: \"kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-kube-api-access-8vr5p\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:35.960816 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:35.960800    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqxs5\" (UniqueName: \"kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-kube-api-access-rqxs5\") pod \"e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") " pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:36.023283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.023254    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:36.044245 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.044211    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:36.353841 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.353811    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"]
Oct 30 05:50:36.355082 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:50:36.355061    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ae4c02_ca60_4ac0_b467_7be612e7faa1.slice/crio-9e7d9c09a845d42bb28dafcc21216fe9f37fdb95d49b6621e4953f6abbcbe3b3 WatchSource:0}: Error finding container 9e7d9c09a845d42bb28dafcc21216fe9f37fdb95d49b6621e4953f6abbcbe3b3: Status 404 returned error can't find the container with id 9e7d9c09a845d42bb28dafcc21216fe9f37fdb95d49b6621e4953f6abbcbe3b3
Oct 30 05:50:36.357780 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.357750    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:36.357876 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.357826    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:36.357876 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.357864    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:36.369485 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:50:36.369462    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod366b7e0f_6d47_4b8e_aeb1_98503d89807a.slice/crio-c4a681a630927bded30181f61f57b0c06749bb510cc3d4dd9dbde355fad25f0a WatchSource:0}: Error finding container c4a681a630927bded30181f61f57b0c06749bb510cc3d4dd9dbde355fad25f0a: Status 404 returned error can't find the container with id c4a681a630927bded30181f61f57b0c06749bb510cc3d4dd9dbde355fad25f0a
Oct 30 05:50:36.372015 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.371984    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:36.372110 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.372050    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:36.372110 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.372081    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:36.372886 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.372866    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"]
Oct 30 05:50:36.433167 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.433124    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:36.455948 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.455916    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" event={"ID":"d7ae4c02-ca60-4ac0-b467-7be612e7faa1","Type":"ContainerStarted","Data":"9e7d9c09a845d42bb28dafcc21216fe9f37fdb95d49b6621e4953f6abbcbe3b3"}
Oct 30 05:50:36.457972 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.457952    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" event={"ID":"366b7e0f-6d47-4b8e-aeb1-98503d89807a","Type":"ContainerStarted","Data":"c4a681a630927bded30181f61f57b0c06749bb510cc3d4dd9dbde355fad25f0a"}
Oct 30 05:50:36.510958 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.510920    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"]
Oct 30 05:50:36.555107 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.555066    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.574837 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.574808    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"]
Oct 30 05:50:36.649063 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.648793    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrlzk\" (UniqueName: \"kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-kube-api-access-lrlzk\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.649250 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.649070    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-credential-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.649250 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.649157    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-certs\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.649250 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.649208    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452c383a-8951-4d4e-abe9-2ba633bed28f-istiod-ca-cert\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.649250 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.649246    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-data\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.649452 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.649288    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-token\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.649508 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.649455    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-envoy\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.649559 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.649507    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.649559 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.649537    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-podinfo\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.751327 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.751279    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrlzk\" (UniqueName: \"kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-kube-api-access-lrlzk\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.751847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.751368    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-credential-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.751847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.751437    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-certs\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.751847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.751482    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452c383a-8951-4d4e-abe9-2ba633bed28f-istiod-ca-cert\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.751847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.751514    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-data\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.751847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.751553    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-token\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.751847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.751766    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-envoy\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.751847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.751820    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.751847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.751846    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-podinfo\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.752236 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.751989    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-certs\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.752285 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.752264    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452c383a-8951-4d4e-abe9-2ba633bed28f-istiod-ca-cert\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.752530 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.752507    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-data\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.752731 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.751769    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-credential-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.752793 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.752751    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.754881 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.754858    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-podinfo\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.754881 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.754865    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-envoy\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.780724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.780688    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrlzk\" (UniqueName: \"kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-kube-api-access-lrlzk\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.782619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.782561    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-token\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:36.897499 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:36.897469    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:37.251615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:37.247823    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"]
Oct 30 05:50:37.271631 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:37.271189    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:37.271631 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:37.271279    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:37.271631 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:37.271323    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:50:37.485673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:37.485636    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" event={"ID":"d7ae4c02-ca60-4ac0-b467-7be612e7faa1","Type":"ContainerStarted","Data":"39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c"}
Oct 30 05:50:37.487945 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:37.487835    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" event={"ID":"366b7e0f-6d47-4b8e-aeb1-98503d89807a","Type":"ContainerStarted","Data":"f314207bb87155d59cbdb8e85e49b7f406a5ac9ac3c5d5cc733f526a7c913b89"}
Oct 30 05:50:37.490375 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:37.490134    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" event={"ID":"452c383a-8951-4d4e-abe9-2ba633bed28f","Type":"ContainerStarted","Data":"1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b"}
Oct 30 05:50:37.490375 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:37.490171    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" event={"ID":"452c383a-8951-4d4e-abe9-2ba633bed28f","Type":"ContainerStarted","Data":"1fc9429b6f9252f3bb97fece72a66dd97daf9795f854ba768672cf14426e7fec"}
Oct 30 05:50:37.618192 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:37.618084    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podStartSLOduration=2.618066394 podStartE2EDuration="2.618066394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:50:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:50:37.617126016 +0000 UTC m=+1141.362411346" watchObservedRunningTime="2026-10-30 05:50:37.618066394 +0000 UTC m=+1141.363351726"
Oct 30 05:50:37.621119 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:37.621059    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podStartSLOduration=2.621044953 podStartE2EDuration="2.621044953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:50:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:50:37.567967801 +0000 UTC m=+1141.313253135" watchObservedRunningTime="2026-10-30 05:50:37.621044953 +0000 UTC m=+1141.366330287"
Oct 30 05:50:37.696196 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:37.696140    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podStartSLOduration=1.6961231319999999 podStartE2EDuration="1.696123132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:50:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:50:37.678086395 +0000 UTC m=+1141.423371750" watchObservedRunningTime="2026-10-30 05:50:37.696123132 +0000 UTC m=+1141.441408464"
Oct 30 05:50:37.898107 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:37.897992    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:38.027615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:38.025059    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:38.038602 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:38.038547    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:38.038907 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:38.038862    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:38.048305 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:38.048263    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:38.048800 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:38.048783    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:38.054842 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:38.054798    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:38.055104 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:38.055051    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:38.911616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:38.907648    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:38.911616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:38.907721    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:39.027614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:39.026994    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:39.027614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:39.027053    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:39.046610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:39.046104    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:39.046610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:39.046166    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:39.619613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:39.619130    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/health\": context deadline exceeded"
Oct 30 05:50:39.660614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:39.657706    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:39.686612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:39.686331    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:39.769614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:39.767667    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:39.905678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:39.902030    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:39.905678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:39.902109    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:40.027615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:40.026006    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:40.027615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:40.026065    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:40.048616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:40.046301    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:40.048616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:40.046365    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:40.411644 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:40.410665    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:40.621636 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:40.620654    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:40.646719 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:40.646671    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:50:40.662799 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:40.662687    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:40.663441 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:40.663416    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:50:41.032669 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.031044    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:41.032669 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.031108    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:41.036349 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.036314    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:41.060682 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.041946    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:41.060682 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.041991    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:41.064992 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.062464    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:41.064992 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.062513    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:41.064992 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.063677    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 05:50:41.089008 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.066334    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:50:41.254870 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.254816    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:41.258952 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.258900    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:41.444615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.443702    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:41.456616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.445734    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:41.577572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.539091    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:41.577572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.548272    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:41.627734 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.627689    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:41.672655 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.627959    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:41.672655 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.641877    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:41.672655 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.654752    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:50:41.701677 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.673117    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:41.701677 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.674446    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:50:41.701677 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.675580    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:50:41.971724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.968877    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:41.971724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:41.968954    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:42.086617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.048962    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:42.086617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.049146    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:42.086617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.049327    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:42.086617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.082487    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:42.086617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.085500    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:42.086617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.085823    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:42.086617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.086093    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:50:42.112642 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.112563    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:42.148618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.143775    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:42.148618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.143860    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:42.305018 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.304917    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:42.312797 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.312753    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:50:42.368644 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.351557    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:42.448735 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.448687    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:42.580045 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.579951    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:50:42.590106 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.590064    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:50:42.674626 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.653315    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 05:50:42.694619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.660733    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:42.784648 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.690155    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:42.784648 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.690188    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 05:50:42.789329 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.789302    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:50:42.797124 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.797086    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:50:42.797351 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.797336    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:50:42.815709 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:42.804809    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:43.093736 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.093694    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:43.099654 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.095576    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:43.101147 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.101099    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 05:50:43.109046 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.109021    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:50:43.126190 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.126145    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:43.290313 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.290282    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:43.291196 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.126531    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/health\": context deadline exceeded"
Oct 30 05:50:43.291196 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.188255    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:43.291414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.147695    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:43.291414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.291278    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:43.291414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.161754    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:43.291414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.291334    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:43.304614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.301123    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:43.304614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.301178    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:43.317676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.317638    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:43.502019 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.501872    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:43.532754 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.531744    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:43.584942 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.584895    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:43.628616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.616659    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:43.765690 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.735684    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 05:50:43.845699 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.799904    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:43.848879 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.805043    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 05:50:43.868336 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.805147    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:50:43.923620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.920023    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:43.923620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.920092    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:43.966696 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:43.966665    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:50:44.025193 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.025086    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:44.025193 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.025158    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:44.055911 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.055877    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:44.056146 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.056119    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:44.096679 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.094769    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 05:50:44.102662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.100669    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:44.102662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.101508    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:44.228614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.222110    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:50:44.323441 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.323219    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:44.323762 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.323738    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" podUID="dafc9b85-538c-4f66-8578-012bad182b0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:44.323937 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.323304    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:44.367829 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.367686    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:44.542358 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.542306    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:44.542658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.542627    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:44.543052 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.543027    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 05:50:44.588352 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.587740    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:44.588352 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.588181    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:44.588352 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.588224    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:44.671926 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.671882    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:44.883674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.870524    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:44.892647 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.873674    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:44.897619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.873710    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 05:50:44.911920 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.911883    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:44.965240 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.965198    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:44.995199 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:44.969712    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 05:50:45.032040 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.032005    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:45.032849 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.032817    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:45.086617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.061280    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:45.087454 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.087414    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:45.113277 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.113237    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:50:45.114790 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.114345    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.115696 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.114385    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.243788 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.243748    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:45.244090 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.244061    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.324565 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.324510    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.374776 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.374728    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.458809 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.458761    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:45.459088 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.459043    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.519862 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.519773    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:45.520127 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.520095    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.520565 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.520545    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:45.520756 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.520736    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.547755 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.547710    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.589837 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.589701    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:45.589837 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.589771    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.590582 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.590170    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:50:45.681917 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.681858    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:50:45.684035 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.683986    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.885807 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.885690    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:50:45.918768 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.918712    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.919410 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.919379    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:45.919612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.919569    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.933212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.933177    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:45.933486 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.933458    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:45.998621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.996723    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:45.998621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:45.997919    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:50:46.034337 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.034304    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:46.034647 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.034618    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:46.047176 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.047141    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:46.047490 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.047440    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:46.084369 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.084332    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:46.084729 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.084698    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:46.116380 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.115782    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:46.117029 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.116998    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:46.118677 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.118649    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:46.369617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.329403    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:46.396671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.391742    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:46.650728 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.566893    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:46.776616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.615677    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:50:46.776616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.693749    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:46.776616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.694487    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:46.847617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.741774    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:50:46.847617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.829915    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:46.914759 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.914710    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:46.984762 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.984714    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:47.036614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.987249    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:47.036614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:46.987307    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:47.036614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.000923    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 05:50:47.064616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.038970    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:47.065147 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.065113    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:47.240709 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.119718    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:50:47.250605 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.119803    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:47.251321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.119833    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 05:50:47.251740 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.130333    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:47.251911 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.251882    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:47.316703 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.308366    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:50:47.368231 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.352744    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 05:50:47.515672 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.414756    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:47.685161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.685097    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:47.820667 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.815794    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:50:47.834628 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.831429    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 05:50:47.863617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.835047    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:50:47.863617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.835299    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 05:50:47.918616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.910275    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:47.918616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.910335    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:47.919294 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:47.919254    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:50:48.104673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.102054    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 05:50:48.104673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.103632    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": context deadline exceeded"
Oct 30 05:50:48.105266 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.105240    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:48.105455 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.105430    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:48.109760 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.109731    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:48.109954 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.109932    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:48.113839 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.110650    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:48.290193 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.248547    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:48.294751 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.251189    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 05:50:48.313723 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.255681    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:48.516684 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.502528    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:48.517449 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.508686    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 05:50:48.675622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.625578    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:50:48.783685 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.783438    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:50:48.819627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.816212    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:48.843610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.839659    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:48.843610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.840899    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 05:50:48.852288 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.851750    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:50:48.948243 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.948199    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:48.959988 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.954836    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:48.965956 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:48.961941    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:49.317612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.044267    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:49.317612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.314409    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:49.317612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.228683    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:50:49.317612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.066792    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:49.317612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.314500    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:49.317612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.315467    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:50:49.318741 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.318698    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:50:49.357072 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.355008    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:50:49.391167 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.391124    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 05:50:49.445940 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.395858    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:49.511050 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.511016    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:49.564659 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.562760    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:49.604853 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.604740    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:49.738618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.711909    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": context deadline exceeded"
Oct 30 05:50:49.850413 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.850370    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:49.855178 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.855068    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:49.907774 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.900074    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:49.907774 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.900135    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:49.909268 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:49.909238    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:50.020178 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.020137    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:50.039486 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.039453    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:50.039743 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.039720    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:50.100207 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.073943    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:50.100207 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.074176    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:50.103621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.100563    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:50.309616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.299716    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:50:50.331625 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.328012    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:50:50.331625 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.329030    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 05:50:50.333128 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.332255    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 05:50:50.421732 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.421690    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:50.517182 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.517000    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:50:50.634787 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.634681    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:50:50.779316 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.779263    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:50.793979 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.793948    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:50:50.822984 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.822958    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:50:50.835893 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.835865    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:50:50.874819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.874770    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:50:50.897544 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.897449    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:50:50.899237 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.899198    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:50.899459 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.899435    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:50.901917 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:50.901897    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:50:51.033307 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:51.033280    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:51.036252 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:51.036226    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 05:50:51.036446 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:51.036424    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 05:50:51.045643 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:51.045613    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:51.069856 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:51.069818    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 05:50:51.070052 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:51.069884    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 05:50:51.660675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:51.658557    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:51.898313 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:51.898277    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:51.899719 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:51.899692    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Oct 30 05:50:51.899936 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:51.899909    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Oct 30 05:50:52.109793 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:52.109705    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:52.117820 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:52.117797    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 05:50:52.338988 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:52.336147    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:52.338988 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:52.338580    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 05:50:52.902318 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:52.902288    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:52.903426 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:52.903409    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:50:53.323085 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:53.323047    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:54.992608 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:54.992562    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:56.553934 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:56.553897    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:58.047198 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:58.047148    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:50:59.635207 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:50:59.635110    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:01.125116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:01.125079    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:02.554020 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:02.553917    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:51:02.747079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:02.747046    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:04.311312 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:04.311278    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:06.975614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:06.964974    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:07.012118 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:07.012078    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:51:07.122427 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:07.122379    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:07.296785 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:07.296682    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:07.325943 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:07.325898    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:07.402616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:07.398860    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:07.471617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:07.470977    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 05:51:07.653667 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:07.653583    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:07.654689 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:07.654659    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:07.664656 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:07.655453    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:51:07.873615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:07.865281    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:07.890792 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:07.878927    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:07.978784 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:07.978682    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": context deadline exceeded"
Oct 30 05:51:08.017818 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.017777    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:08.041919 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.041887    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:51:08.113434 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.113392    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 05:51:08.140828 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.140788    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:08.214435 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.191851    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:51:08.234438 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.215369    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:51:08.365614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.362241    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:08.406302 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.364678    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:08.456619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.439665    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Oct 30 05:51:08.501674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.498866    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 05:51:08.659613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.634831    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:51:08.712652 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.709979    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:51:08.712652 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.710066    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:51:08.712652 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.710104    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:51:08.712652 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.660298    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:08.712652 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.692966    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:51:08.716620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.713857    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:51:08.716620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.713907    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:51:08.901675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.897671    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:08.901675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.898661    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 05:51:08.928726 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.928694    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:51:08.929014 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.928997    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:51:09.010616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.993884    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:51:09.010616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:08.993945    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:09.198679 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.049892    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:51:09.316617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.316402    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:51:09.316617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.124688    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 05:51:09.339617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.319720    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:51:09.515674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.510040    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:09.600616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.572735    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:09.639067 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.602568    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:09.917677 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.896699    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:51:09.917677 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.610400    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 05:51:09.936426 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.726465    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Oct 30 05:51:09.936426 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.928665    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:09.936426 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.734216    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/health\": context deadline exceeded"
Oct 30 05:51:09.936426 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.743145    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": context deadline exceeded"
Oct 30 05:51:09.936426 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.734528    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:09.936426 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.742921    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:51:09.946627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.920703    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 05:51:09.989615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:09.983926    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:10.317676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:10.317031    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:10.366617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:10.362528    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:51:10.367246 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:10.367209    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 05:51:10.636346 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:10.636243    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 05:51:10.885616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:10.799989    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 05:51:10.887180 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:10.808055    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 05:51:10.901351 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:10.899936    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:10.928672 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:10.928637    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:10.929097 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:10.929053    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:10.955726 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:10.942298    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 05:51:10.955726 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:10.951896    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:10.955726 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:10.952177    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 05:51:11.274628 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.249685    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:11.395770 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.395718    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:11.463111 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.427902    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:51:11.526666 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.498040    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:51:11.541070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.541040    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:51:11.650612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.645695    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:51:11.688060 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.688031    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:51:11.719485 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.715488    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:51:11.719485 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.715538    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:51:11.719485 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.715562    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:51:11.719485 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.715631    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:51:11.719485 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.715657    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:51:11.719485 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.715683    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:51:11.719485 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.717967    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:51:11.751374 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.751333    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:51:11.856377 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:11.856274    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:13.510661 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:13.510621    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:15.312824 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:15.312791    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:16.908487 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:16.908443    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:17.704404 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:17.704364    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9213-183013"]
Oct 30 05:51:17.709734 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:17.709710    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9213-183013"
Oct 30 05:51:17.726056 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:17.726020    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9213-183013"]
Oct 30 05:51:17.735501 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:17.735470    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4m56v\""
Oct 30 05:51:17.785782 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:17.785746    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6l5h\" (UniqueName: \"kubernetes.io/projected/8d6f49aa-3e06-4423-99e4-eab4f81f2222-kube-api-access-l6l5h\") pod \"test-curl-9213-183013\" (UID: \"8d6f49aa-3e06-4423-99e4-eab4f81f2222\") " pod="opendatahub/test-curl-9213-183013"
Oct 30 05:51:17.887658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:17.887617    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6l5h\" (UniqueName: \"kubernetes.io/projected/8d6f49aa-3e06-4423-99e4-eab4f81f2222-kube-api-access-l6l5h\") pod \"test-curl-9213-183013\" (UID: \"8d6f49aa-3e06-4423-99e4-eab4f81f2222\") " pod="opendatahub/test-curl-9213-183013"
Oct 30 05:51:17.913057 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:17.913021    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6l5h\" (UniqueName: \"kubernetes.io/projected/8d6f49aa-3e06-4423-99e4-eab4f81f2222-kube-api-access-l6l5h\") pod \"test-curl-9213-183013\" (UID: \"8d6f49aa-3e06-4423-99e4-eab4f81f2222\") " pod="opendatahub/test-curl-9213-183013"
Oct 30 05:51:18.023504 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:18.023430    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9213-183013"
Oct 30 05:51:18.329148 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:18.329109    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9213-183013"]
Oct 30 05:51:18.329575 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:51:18.329553    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d6f49aa_3e06_4423_99e4_eab4f81f2222.slice/crio-0980a69ad0f7d3115fd7a058133a6e32158e80065bd05877013e1f3dffd40dbf WatchSource:0}: Error finding container 0980a69ad0f7d3115fd7a058133a6e32158e80065bd05877013e1f3dffd40dbf: Status 404 returned error can't find the container with id 0980a69ad0f7d3115fd7a058133a6e32158e80065bd05877013e1f3dffd40dbf
Oct 30 05:51:18.331686 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:18.331669    5066 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 05:51:18.598839 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:18.598748    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:18.986503 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:18.986472    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9213-183013" event={"ID":"8d6f49aa-3e06-4423-99e4-eab4f81f2222","Type":"ContainerStarted","Data":"0980a69ad0f7d3115fd7a058133a6e32158e80065bd05877013e1f3dffd40dbf"}
Oct 30 05:51:19.995734 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:19.995694    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9213-183013" event={"ID":"8d6f49aa-3e06-4423-99e4-eab4f81f2222","Type":"ContainerStarted","Data":"d84c5d2a4acaa1f00fadcf012041bc31ab1eef733f2eefd1a6964be865acd310"}
Oct 30 05:51:20.037831 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:20.037769    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9213-183013" podStartSLOduration=2.446610233 podStartE2EDuration="3.037750819s" totalImagesPullingTime="591.140586ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:51:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:51:20.037558232 +0000 UTC m=+1183.782843564" watchObservedRunningTime="2026-10-30 05:51:20.037750819 +0000 UTC m=+1183.783036151"
Oct 30 05:51:20.314291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:20.314191    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:20.953243 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:20.953211    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9213-183013"]
Oct 30 05:51:20.956980 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:20.956955    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9213-183013"]
Oct 30 05:51:21.117724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:21.117693    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9213-1ea43b"]
Oct 30 05:51:21.118484 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:21.118458    5066 cpu_manager.go:414] "containerMap: removing container" podUID="8d6f49aa-3e06-4423-99e4-eab4f81f2222" containerName="test-curl-9213-183013"
Oct 30 05:51:21.118484 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:21.118480    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d6f49aa-3e06-4423-99e4-eab4f81f2222" containerName="test-curl-9213-183013"
Oct 30 05:51:21.118667 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:21.118611    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d6f49aa-3e06-4423-99e4-eab4f81f2222" containerName="test-curl-9213-183013"
Oct 30 05:51:21.122382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:21.122367    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9213-1ea43b"
Oct 30 05:51:21.139885 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:21.139859    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9213-1ea43b"]
Oct 30 05:51:21.246422 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:21.246341    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbsvc\" (UniqueName: \"kubernetes.io/projected/7ddd4508-f1f4-49e7-ada8-2026bb7b6864-kube-api-access-bbsvc\") pod \"test-curl-9213-1ea43b\" (UID: \"7ddd4508-f1f4-49e7-ada8-2026bb7b6864\") " pod="opendatahub/test-curl-9213-1ea43b"
Oct 30 05:51:21.347887 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:21.347852    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbsvc\" (UniqueName: \"kubernetes.io/projected/7ddd4508-f1f4-49e7-ada8-2026bb7b6864-kube-api-access-bbsvc\") pod \"test-curl-9213-1ea43b\" (UID: \"7ddd4508-f1f4-49e7-ada8-2026bb7b6864\") " pod="opendatahub/test-curl-9213-1ea43b"
Oct 30 05:51:21.373249 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:21.373221    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbsvc\" (UniqueName: \"kubernetes.io/projected/7ddd4508-f1f4-49e7-ada8-2026bb7b6864-kube-api-access-bbsvc\") pod \"test-curl-9213-1ea43b\" (UID: \"7ddd4508-f1f4-49e7-ada8-2026bb7b6864\") " pod="opendatahub/test-curl-9213-1ea43b"
Oct 30 05:51:21.434518 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:21.434489    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9213-1ea43b"
Oct 30 05:51:21.748743 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:21.748714    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9213-1ea43b"]
Oct 30 05:51:21.752771 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:51:21.752746    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ddd4508_f1f4_49e7_ada8_2026bb7b6864.slice/crio-03207a64620718f3de4e208d7a1d8a30a7a566130ef9ef9f3e8f71d5a90a3564 WatchSource:0}: Error finding container 03207a64620718f3de4e208d7a1d8a30a7a566130ef9ef9f3e8f71d5a90a3564: Status 404 returned error can't find the container with id 03207a64620718f3de4e208d7a1d8a30a7a566130ef9ef9f3e8f71d5a90a3564
Oct 30 05:51:21.920657 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:21.912896    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:22.014990 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:22.014869    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9213-183013" podUID="8d6f49aa-3e06-4423-99e4-eab4f81f2222" containerName="test-curl-9213-183013" containerID="cri-o://d84c5d2a4acaa1f00fadcf012041bc31ab1eef733f2eefd1a6964be865acd310" gracePeriod=2
Oct 30 05:51:22.015267 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:22.015164    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9213-1ea43b" event={"ID":"7ddd4508-f1f4-49e7-ada8-2026bb7b6864","Type":"ContainerStarted","Data":"66c284d115c4a1041938345c8f224304827dcbf1d30b2643acdb7abbb0c4ed86"}
Oct 30 05:51:22.015267 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:22.015242    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9213-1ea43b" event={"ID":"7ddd4508-f1f4-49e7-ada8-2026bb7b6864","Type":"ContainerStarted","Data":"03207a64620718f3de4e208d7a1d8a30a7a566130ef9ef9f3e8f71d5a90a3564"}
Oct 30 05:51:22.058766 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:22.058713    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9213-1ea43b" podStartSLOduration=1.058698133 podStartE2EDuration="1.058698133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:51:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:51:22.055707157 +0000 UTC m=+1185.800992489" watchObservedRunningTime="2026-10-30 05:51:22.058698133 +0000 UTC m=+1185.803983464"
Oct 30 05:51:22.071052 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:22.071018    5066 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9213-183013\" is forbidden: User \"system:node:ip-10-0-136-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-173.ec2.internal' and this object" podUID="8d6f49aa-3e06-4423-99e4-eab4f81f2222" pod="opendatahub/test-curl-9213-183013"
Oct 30 05:51:22.086458 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:22.086426    5066 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9213-183013\" is forbidden: User \"system:node:ip-10-0-136-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-173.ec2.internal' and this object" podUID="8d6f49aa-3e06-4423-99e4-eab4f81f2222" pod="opendatahub/test-curl-9213-183013"
Oct 30 05:51:22.894711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:22.894674    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9213-1ea43b"]
Oct 30 05:51:22.901525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:22.899828    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9213-1ea43b"]
Oct 30 05:51:23.597048 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:23.597002    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:24.029297 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:24.029266    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9213-1ea43b" podUID="7ddd4508-f1f4-49e7-ada8-2026bb7b6864" containerName="test-curl-9213-1ea43b" containerID="cri-o://66c284d115c4a1041938345c8f224304827dcbf1d30b2643acdb7abbb0c4ed86" gracePeriod=2
Oct 30 05:51:24.044863 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:24.044811    5066 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9213-1ea43b\" is forbidden: User \"system:node:ip-10-0-136-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-173.ec2.internal' and this object" podUID="7ddd4508-f1f4-49e7-ada8-2026bb7b6864" pod="opendatahub/test-curl-9213-1ea43b"
Oct 30 05:51:24.208204 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:24.208177    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9213-183013"
Oct 30 05:51:24.223866 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:24.223835    5066 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9213-1ea43b\" is forbidden: User \"system:node:ip-10-0-136-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-173.ec2.internal' and this object" podUID="7ddd4508-f1f4-49e7-ada8-2026bb7b6864" pod="opendatahub/test-curl-9213-1ea43b"
Oct 30 05:51:24.292972 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:24.292899    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d6f49aa-3e06-4423-99e4-eab4f81f2222-kube-api-access-l6l5h\" (UniqueName: \"kubernetes.io/projected/8d6f49aa-3e06-4423-99e4-eab4f81f2222-kube-api-access-l6l5h\") pod \"8d6f49aa-3e06-4423-99e4-eab4f81f2222\" (UID: \"8d6f49aa-3e06-4423-99e4-eab4f81f2222\") "
Oct 30 05:51:24.295187 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:24.295160    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d6f49aa-3e06-4423-99e4-eab4f81f2222-kube-api-access-l6l5h" pod "8d6f49aa-3e06-4423-99e4-eab4f81f2222" (UID: "8d6f49aa-3e06-4423-99e4-eab4f81f2222"). InnerVolumeSpecName "kube-api-access-l6l5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:51:24.395637 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:24.395583    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6l5h\" (UniqueName: \"kubernetes.io/projected/8d6f49aa-3e06-4423-99e4-eab4f81f2222-kube-api-access-l6l5h\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:51:24.403619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:24.403569    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d6f49aa-3e06-4423-99e4-eab4f81f2222" path="/var/lib/kubelet/pods/8d6f49aa-3e06-4423-99e4-eab4f81f2222/volumes"
Oct 30 05:51:25.037285 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:25.037250    5066 generic.go:356] "Generic (PLEG): container finished" podID="8d6f49aa-3e06-4423-99e4-eab4f81f2222" containerID="d84c5d2a4acaa1f00fadcf012041bc31ab1eef733f2eefd1a6964be865acd310" exitCode=137
Oct 30 05:51:25.037858 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:25.037294    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9213-183013"
Oct 30 05:51:25.037858 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:25.037311    5066 scope.go:122] "RemoveContainer" containerID="d84c5d2a4acaa1f00fadcf012041bc31ab1eef733f2eefd1a6964be865acd310"
Oct 30 05:51:25.047910 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:25.047893    5066 scope.go:122] "RemoveContainer" containerID="d84c5d2a4acaa1f00fadcf012041bc31ab1eef733f2eefd1a6964be865acd310"
Oct 30 05:51:25.048196 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:25.048175    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d84c5d2a4acaa1f00fadcf012041bc31ab1eef733f2eefd1a6964be865acd310\": container with ID starting with d84c5d2a4acaa1f00fadcf012041bc31ab1eef733f2eefd1a6964be865acd310 not found: ID does not exist" containerID="d84c5d2a4acaa1f00fadcf012041bc31ab1eef733f2eefd1a6964be865acd310"
Oct 30 05:51:25.048265 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:25.048208    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84c5d2a4acaa1f00fadcf012041bc31ab1eef733f2eefd1a6964be865acd310"} err="failed to get container status \"d84c5d2a4acaa1f00fadcf012041bc31ab1eef733f2eefd1a6964be865acd310\": rpc error: code = NotFound desc = could not find container \"d84c5d2a4acaa1f00fadcf012041bc31ab1eef733f2eefd1a6964be865acd310\": container with ID starting with d84c5d2a4acaa1f00fadcf012041bc31ab1eef733f2eefd1a6964be865acd310 not found: ID does not exist"
Oct 30 05:51:25.098197 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:25.098159    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:25.414069 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:25.414006    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:25.584254 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:25.584225    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:26.125842 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:26.125792    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:26.356734 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:26.356709    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9213-1ea43b"
Oct 30 05:51:26.457177 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:26.457006    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ddd4508-f1f4-49e7-ada8-2026bb7b6864-kube-api-access-bbsvc\" (UniqueName: \"kubernetes.io/projected/7ddd4508-f1f4-49e7-ada8-2026bb7b6864-kube-api-access-bbsvc\") pod \"7ddd4508-f1f4-49e7-ada8-2026bb7b6864\" (UID: \"7ddd4508-f1f4-49e7-ada8-2026bb7b6864\") "
Oct 30 05:51:26.478255 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:26.478196    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ddd4508-f1f4-49e7-ada8-2026bb7b6864-kube-api-access-bbsvc" pod "7ddd4508-f1f4-49e7-ada8-2026bb7b6864" (UID: "7ddd4508-f1f4-49e7-ada8-2026bb7b6864"). InnerVolumeSpecName "kube-api-access-bbsvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:51:26.598114 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:26.598023    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ddd4508-f1f4-49e7-ada8-2026bb7b6864" path="/var/lib/kubelet/pods/7ddd4508-f1f4-49e7-ada8-2026bb7b6864/volumes"
Oct 30 05:51:26.602524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:26.598943    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbsvc\" (UniqueName: \"kubernetes.io/projected/7ddd4508-f1f4-49e7-ada8-2026bb7b6864-kube-api-access-bbsvc\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:51:27.117910 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:27.116857    5066 generic.go:356] "Generic (PLEG): container finished" podID="7ddd4508-f1f4-49e7-ada8-2026bb7b6864" containerID="66c284d115c4a1041938345c8f224304827dcbf1d30b2643acdb7abbb0c4ed86" exitCode=137
Oct 30 05:51:27.117910 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:27.116950    5066 scope.go:122] "RemoveContainer" containerID="66c284d115c4a1041938345c8f224304827dcbf1d30b2643acdb7abbb0c4ed86"
Oct 30 05:51:27.117910 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:27.117117    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9213-1ea43b"
Oct 30 05:51:27.284099 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:27.279759    5066 scope.go:122] "RemoveContainer" containerID="66c284d115c4a1041938345c8f224304827dcbf1d30b2643acdb7abbb0c4ed86"
Oct 30 05:51:27.304066 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:27.304023    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66c284d115c4a1041938345c8f224304827dcbf1d30b2643acdb7abbb0c4ed86\": container with ID starting with 66c284d115c4a1041938345c8f224304827dcbf1d30b2643acdb7abbb0c4ed86 not found: ID does not exist" containerID="66c284d115c4a1041938345c8f224304827dcbf1d30b2643acdb7abbb0c4ed86"
Oct 30 05:51:27.304345 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:27.304309    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66c284d115c4a1041938345c8f224304827dcbf1d30b2643acdb7abbb0c4ed86"} err="failed to get container status \"66c284d115c4a1041938345c8f224304827dcbf1d30b2643acdb7abbb0c4ed86\": rpc error: code = NotFound desc = could not find container \"66c284d115c4a1041938345c8f224304827dcbf1d30b2643acdb7abbb0c4ed86\": container with ID starting with 66c284d115c4a1041938345c8f224304827dcbf1d30b2643acdb7abbb0c4ed86 not found: ID does not exist"
Oct 30 05:51:27.350629 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:27.339857    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:27.905731 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:27.905684    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 05:51:27.963974 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:27.963937    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": context deadline exceeded"
Oct 30 05:51:27.970995 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:27.964214    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:27.971491 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:27.965394    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:51:27.993021 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:27.992977    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:28.208679 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.191118    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:28.311730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.311685    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:51:28.331728 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.331686    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:28.414616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.411647    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:28.414616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.411808    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 05:51:28.690617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.667316    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 05:51:28.719614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.718853    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:28.740944 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.740854    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:28.754614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.742724    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:28.916744 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.916700    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 05:51:28.959321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.918560    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:28.961356 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.961330    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:51:28.972703 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.971675    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:28.972703 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.971816    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:51:28.972703 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.972061    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:51:28.972703 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:28.972115    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:51:29.016246 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.016203    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:29.016742 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.016719    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:51:29.342701 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.321978    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:51:29.343319 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.343292    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:51:29.343565 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.343534    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:29.351627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.346002    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:51:29.515617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.463851    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:29.515617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.465670    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:51:29.635669 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.612850    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:29.687400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.686187    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 05:51:29.714321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.688537    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:51:29.714321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.712955    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:51:29.714321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.713018    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:29.762731 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.735164    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:29.762731 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.736346    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": context deadline exceeded"
Oct 30 05:51:29.762731 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.736459    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:51:29.768892 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.767663    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:29.812674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.795176    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:51:29.812674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.796818    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 05:51:29.812674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.797190    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:51:29.959735 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.959686    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:29.960340 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.960297    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:29.960863 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.960841    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:51:29.996612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:29.985488    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:30.195615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.022445    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:30.195615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.022495    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Oct 30 05:51:30.195615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.022532    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:51:30.195615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.038428    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:51:30.255753 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.255672    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:51:30.327676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.313262    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:51:30.333990 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.327987    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:30.367629 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.356714    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:30.367629 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.366012    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:51:30.596132 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.596029    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:30.615646 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.605654    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:51:30.615646 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.605784    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:30.655355 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.655322    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:51:30.655748 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.655730    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:51:30.655971 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.655956    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:51:30.656163 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.656150    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:51:30.688622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.687222    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 05:51:30.696885 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.696734    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:51:30.698937 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.698172    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:51:30.727202 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.722331    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:51:30.727202 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.725679    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:51:30.761551 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:30.761502    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:32.241488 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:32.241457    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:32.450301 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:32.450269    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:32.562462 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:32.562375    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:51:32.610521 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:32.610449    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:32.789162 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:32.789126    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:33.437583 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:33.437551    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:33.595406 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:33.595372    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:33.752864 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:33.752773    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:34.027850 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.027767    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:34.066188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.066152    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"]
Oct 30 05:51:34.067258 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.067237    5066 cpu_manager.go:414] "containerMap: removing container" podUID="7ddd4508-f1f4-49e7-ada8-2026bb7b6864" containerName="test-curl-9213-1ea43b"
Oct 30 05:51:34.067313 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.067263    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ddd4508-f1f4-49e7-ada8-2026bb7b6864" containerName="test-curl-9213-1ea43b"
Oct 30 05:51:34.067442 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.067429    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ddd4508-f1f4-49e7-ada8-2026bb7b6864" containerName="test-curl-9213-1ea43b"
Oct 30 05:51:34.075219 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.073556    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.092640 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.091714    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"]
Oct 30 05:51:34.106677 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.106632    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-istio-token\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.106876 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.106721    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5jdf\" (UniqueName: \"kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-kube-api-access-q5jdf\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.106876 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.106784    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-certs\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.106876 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.106815    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-credential-socket\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.107044 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.106958    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-data\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.107044 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.106985    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-709c4831-openshift-default-dockercfg-2bs4j\""
Oct 30 05:51:34.107044 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.107012    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-envoy\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.107240 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.107069    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1718d33e-9c55-4afc-a897-5251337302d9-istio-podinfo\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.107240 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.107166    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1718d33e-9c55-4afc-a897-5251337302d9-istiod-ca-cert\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.107240 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.107212    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-socket\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.196620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.194028    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:34.212619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.211707    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-socket\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.212619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.211774    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-istio-token\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.212619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.211834    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5jdf\" (UniqueName: \"kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-kube-api-access-q5jdf\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.212619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.211895    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-certs\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.212619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.211929    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-credential-socket\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.212619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.212087    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-data\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.212619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.212143    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-envoy\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.212619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.212202    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1718d33e-9c55-4afc-a897-5251337302d9-istio-podinfo\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.212619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.212302    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1718d33e-9c55-4afc-a897-5251337302d9-istiod-ca-cert\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.213263 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.213233    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1718d33e-9c55-4afc-a897-5251337302d9-istiod-ca-cert\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.213618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.213520    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-socket\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.217616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.214272    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-certs\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.217616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.214508    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-credential-socket\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.224618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.222822    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-data\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.270660 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.263429    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5jdf\" (UniqueName: \"kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-kube-api-access-q5jdf\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.274252 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.272313    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1718d33e-9c55-4afc-a897-5251337302d9-istio-podinfo\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.303669 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.278413    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-envoy\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.303669 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.278812    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-istio-token\") pod \"e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") " pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.412872 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.411432    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:34.587693 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.587185    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:34.764660 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.764601    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"]
Oct 30 05:51:34.767473 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:51:34.766950    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1718d33e_9c55_4afc_a897_5251337302d9.slice/crio-cf832aefd906e580170c1539d1cdc1feaa5a7d3739f417e1823f6fd5c9cc3cdf WatchSource:0}: Error finding container cf832aefd906e580170c1539d1cdc1feaa5a7d3739f417e1823f6fd5c9cc3cdf: Status 404 returned error can't find the container with id cf832aefd906e580170c1539d1cdc1feaa5a7d3739f417e1823f6fd5c9cc3cdf
Oct 30 05:51:34.770490 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.770456    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:34.770646 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.770524    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:34.770646 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:34.770552    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:35.681344 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:35.681301    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq" event={"ID":"1718d33e-9c55-4afc-a897-5251337302d9","Type":"ContainerStarted","Data":"68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5"}
Oct 30 05:51:35.681751 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:35.681388    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq" event={"ID":"1718d33e-9c55-4afc-a897-5251337302d9","Type":"ContainerStarted","Data":"cf832aefd906e580170c1539d1cdc1feaa5a7d3739f417e1823f6fd5c9cc3cdf"}
Oct 30 05:51:35.749935 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:35.749885    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq" podStartSLOduration=1.749869909 podStartE2EDuration="1.749869909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:51:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:51:35.748955717 +0000 UTC m=+1199.494241051" watchObservedRunningTime="2026-10-30 05:51:35.749869909 +0000 UTC m=+1199.495155241"
Oct 30 05:51:35.825061 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:35.825027    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:36.064315 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.064231    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"]
Oct 30 05:51:36.068765 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.068726    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.106003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.105969    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"]
Oct 30 05:51:36.147983 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.147946    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-data\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.148216 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.148178    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-envoy\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.148343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.148269    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-certs\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.148410 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.148383    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khs6v\" (UniqueName: \"kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-kube-api-access-khs6v\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.148485 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.148463    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-credential-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.148651 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.148496    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414bfcf3-b693-454e-be82-464207f3ede4-istio-podinfo\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.148651 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.148519    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414bfcf3-b693-454e-be82-464207f3ede4-istiod-ca-cert\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.149059 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.148583    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-istio-token\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.149193 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.149179    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.250259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.250226    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-istio-token\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.250431 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.250323    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.250431 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.250378    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-data\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.250431 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.250408    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-envoy\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.250678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.250458    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-certs\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.250678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.250508    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khs6v\" (UniqueName: \"kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-kube-api-access-khs6v\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.250678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.250574    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-credential-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.250678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.250633    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414bfcf3-b693-454e-be82-464207f3ede4-istio-podinfo\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.250678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.250656    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414bfcf3-b693-454e-be82-464207f3ede4-istiod-ca-cert\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.250938 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.250775    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.250938 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.250854    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-certs\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.250938 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.250913    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-data\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.251344 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.251040    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-credential-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.251344 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.251322    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414bfcf3-b693-454e-be82-464207f3ede4-istiod-ca-cert\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.253314 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.253290    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-envoy\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.253314 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.253309    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414bfcf3-b693-454e-be82-464207f3ede4-istio-podinfo\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.276751 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.276720    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-istio-token\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.276890 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.276873    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khs6v\" (UniqueName: \"kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-kube-api-access-khs6v\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.389801 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.389724    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:36.412603 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.412557    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:36.419497 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.419469    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:36.559288 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.559195    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:51:36.570210 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.570172    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:51:36.683698 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.683662    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:36.688090 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.688063    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:36.689202 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.689182    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:51:36.727634 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:51:36.727605    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414bfcf3_b693_454e_be82_464207f3ede4.slice/crio-a4f455843b790f0772f3078b694b3a483fca92ffd6d4052729147d5fe364de31 WatchSource:0}: Error finding container a4f455843b790f0772f3078b694b3a483fca92ffd6d4052729147d5fe364de31: Status 404 returned error can't find the container with id a4f455843b790f0772f3078b694b3a483fca92ffd6d4052729147d5fe364de31
Oct 30 05:51:36.729879 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.729854    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"]
Oct 30 05:51:36.730303 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.730273    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:36.730399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.730336    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:36.730399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.730363    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:36.745620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.745578    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"]
Oct 30 05:51:36.751660 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.751607    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.774229 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.773555    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"]
Oct 30 05:51:36.789955 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.789924    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.796162 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.796132    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"]
Oct 30 05:51:36.803320 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.803292    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"]
Oct 30 05:51:36.848677 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.848643    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:36.871720 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.871679    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-data\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.871915 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.871805    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6660aa46-642d-4798-845c-e70792eaac8c-istio-podinfo\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.871915 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.871836    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-envoy\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.871915 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.871871    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-token\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.872098 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.871915    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6tkn\" (UniqueName: \"kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-kube-api-access-x6tkn\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.872098 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.871957    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-podinfo\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.872098 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.871988    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.872098 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.872029    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istiod-ca-cert\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.872098 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.872067    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.872335 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.872097    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-istio-token\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.872335 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.872129    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6660aa46-642d-4798-845c-e70792eaac8c-istiod-ca-cert\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.872335 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.872187    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-credential-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.872335 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.872262    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-credential-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.872548 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.872395    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlwvg\" (UniqueName: \"kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-kube-api-access-hlwvg\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.872548 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.872444    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-certs\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.872548 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.872476    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-data\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.872548 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.872495    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-envoy\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.872783 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.872558    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-certs\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.974322 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.974238    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-credential-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.974322 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.974280    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-credential-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.974524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.974401    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlwvg\" (UniqueName: \"kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-kube-api-access-hlwvg\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.974524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.974444    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-certs\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.974524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.974476    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-data\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.974524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.974505    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-envoy\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.974761 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.974580    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-certs\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.974761 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.974668    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-data\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.974761 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.974738    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-credential-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.974920 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.974806    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-credential-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.974920 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.974855    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-data\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.975030 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975017    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-data\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.975115 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975089    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6660aa46-642d-4798-845c-e70792eaac8c-istio-podinfo\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.975182 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975116    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-certs\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.975182 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975122    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-envoy\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.975280 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975191    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-token\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.975280 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975242    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6tkn\" (UniqueName: \"kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-kube-api-access-x6tkn\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.975381 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975282    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-podinfo\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.975381 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975316    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.975381 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975357    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istiod-ca-cert\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.975524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975400    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.975524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975431    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-istio-token\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.975524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975465    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6660aa46-642d-4798-845c-e70792eaac8c-istiod-ca-cert\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.975884 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.975857    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-certs\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.976110 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.976086    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.976234 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.976208    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6660aa46-642d-4798-845c-e70792eaac8c-istiod-ca-cert\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.976577 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.976551    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-socket\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.976883 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.976861    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istiod-ca-cert\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.977709 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.977690    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-envoy\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.977800 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.977782    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-envoy\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:36.978117 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.978097    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6660aa46-642d-4798-845c-e70792eaac8c-istio-podinfo\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:36.978624 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:36.978562    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-podinfo\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:37.000864 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.000834    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlwvg\" (UniqueName: \"kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-kube-api-access-hlwvg\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:37.001023 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.000876    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-token\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:37.001430 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.001410    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6tkn\" (UniqueName: \"kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-kube-api-access-x6tkn\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:37.001781 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.001762    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-istio-token\") pod \"e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") " pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:37.136675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.136642    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:37.152126 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.152098    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:37.484872 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.484846    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:37.486800 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.486756    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:37.486911 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.486848    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:37.486911 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.486890    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:37.487328 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.487308    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"]
Oct 30 05:51:37.493564 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.493472    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"]
Oct 30 05:51:37.504470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.504436    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:37.504627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.504508    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:37.504627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.504547    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:37.662964 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.662940    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:37.716802 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.716762    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" event={"ID":"61a0a834-7508-4682-bd91-d4cde3cbd2ad","Type":"ContainerStarted","Data":"583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9"}
Oct 30 05:51:37.755718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.743428    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" event={"ID":"61a0a834-7508-4682-bd91-d4cde3cbd2ad","Type":"ContainerStarted","Data":"4a25b4e660c7c91078e251b3e0fad152bef19719d4695037cad748b51871c7dc"}
Oct 30 05:51:37.779994 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.779957    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" event={"ID":"414bfcf3-b693-454e-be82-464207f3ede4","Type":"ContainerStarted","Data":"6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c"}
Oct 30 05:51:37.780192 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.780024    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" event={"ID":"414bfcf3-b693-454e-be82-464207f3ede4","Type":"ContainerStarted","Data":"a4f455843b790f0772f3078b694b3a483fca92ffd6d4052729147d5fe364de31"}
Oct 30 05:51:37.793060 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.788958    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" event={"ID":"6660aa46-642d-4798-845c-e70792eaac8c","Type":"ContainerStarted","Data":"d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461"}
Oct 30 05:51:37.793060 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.789014    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" event={"ID":"6660aa46-642d-4798-845c-e70792eaac8c","Type":"ContainerStarted","Data":"a1bbce18c791aa06de88d372d11f8b53a8e9ef55f4eac6875475b8a719627d93"}
Oct 30 05:51:37.855137 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.855100    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:37.871514 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.868035    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" podStartSLOduration=1.8680125090000002 podStartE2EDuration="1.868012509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:51:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:51:37.809227059 +0000 UTC m=+1201.554512394" watchObservedRunningTime="2026-10-30 05:51:37.868012509 +0000 UTC m=+1201.613297841"
Oct 30 05:51:37.875877 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.875820    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" podStartSLOduration=1.8758028389999999 podStartE2EDuration="1.875802839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:51:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:51:37.867723664 +0000 UTC m=+1201.613008997" watchObservedRunningTime="2026-10-30 05:51:37.875802839 +0000 UTC m=+1201.621088173"
Oct 30 05:51:37.919455 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:37.919382    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" podStartSLOduration=1.919363722 podStartE2EDuration="1.919363722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:51:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:51:37.916859004 +0000 UTC m=+1201.662144335" watchObservedRunningTime="2026-10-30 05:51:37.919363722 +0000 UTC m=+1201.664649056"
Oct 30 05:51:38.142640 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:38.142533    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:38.142994 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:38.142971    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 05:51:38.143149 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:38.143127    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" podUID="6660aa46-642d-4798-845c-e70792eaac8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 05:51:38.151022 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:38.150990    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:38.170854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:38.170820    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:38.174060 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:38.172183    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 05:51:38.174287 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:38.174261    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 05:51:38.443617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:38.442737    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 05:51:38.443617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:38.442800    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" podUID="414bfcf3-b693-454e-be82-464207f3ede4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 05:51:38.462671 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:38.462542    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:39.143037 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:39.142998    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 05:51:39.143683 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:39.143638    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" podUID="6660aa46-642d-4798-845c-e70792eaac8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 05:51:39.163058 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:39.163021    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 05:51:39.163392 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:39.163363    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 05:51:39.391613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:39.391056    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 05:51:39.391613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:39.391128    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" podUID="414bfcf3-b693-454e-be82-464207f3ede4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 05:51:39.822913 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:39.822866    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:40.013482 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:40.013447    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:40.184667 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:40.174181    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 05:51:40.184667 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:40.174260    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" podUID="6660aa46-642d-4798-845c-e70792eaac8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 05:51:40.234662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:40.221444    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:40.259711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:40.243841    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 05:51:40.259711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:40.243919    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 05:51:40.504281 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:40.495155    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 05:51:40.504281 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:40.495236    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" podUID="414bfcf3-b693-454e-be82-464207f3ede4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 05:51:41.147642 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:41.139984    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 05:51:41.147642 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:41.140048    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" podUID="6660aa46-642d-4798-845c-e70792eaac8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 05:51:41.153116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:41.153073    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 05:51:41.154803 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:41.153370    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 05:51:41.393613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:41.392986    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 05:51:41.393613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:41.393052    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" podUID="414bfcf3-b693-454e-be82-464207f3ede4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 05:51:41.778675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:41.775223    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:41.975619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:41.974252    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:42.137681 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:42.137162    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 05:51:42.137681 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:42.137224    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" podUID="6660aa46-642d-4798-845c-e70792eaac8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 05:51:42.170616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:42.169671    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 05:51:42.170616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:42.169730    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 05:51:42.174613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:42.172076    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:42.406688 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:42.406219    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 05:51:42.406688 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:42.406278    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" podUID="414bfcf3-b693-454e-be82-464207f3ede4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 05:51:43.148621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:43.148086    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 05:51:43.148621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:43.148150    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" podUID="6660aa46-642d-4798-845c-e70792eaac8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 05:51:43.155127 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:43.155004    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 05:51:43.155127 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:43.155065    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 05:51:43.392614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:43.391148    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 05:51:43.392614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:43.391231    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" podUID="414bfcf3-b693-454e-be82-464207f3ede4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 05:51:44.094616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:44.094005    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:44.142656 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:44.138020    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 05:51:44.142656 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:44.138083    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" podUID="6660aa46-642d-4798-845c-e70792eaac8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 05:51:44.166534 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:44.165825    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 05:51:44.166534 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:44.165879    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 05:51:44.391662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:44.391124    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 05:51:44.391662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:44.391185    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" podUID="414bfcf3-b693-454e-be82-464207f3ede4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 05:51:45.138088 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:45.138050    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 05:51:45.138506 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:45.138138    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" podUID="6660aa46-642d-4798-845c-e70792eaac8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 05:51:45.152582 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:45.152546    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 05:51:45.152774 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:45.152646    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 05:51:45.396191 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:45.396102    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:45.396405 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:45.396375    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:45.397369 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:45.397346    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 05:51:46.019246 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:46.019215    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:46.611290 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:46.611104    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:46.612851 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:46.612702    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:46.617265 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:46.617236    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 05:51:46.675773 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:46.675743    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:46.676578 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:46.676553    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:46.677714 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:46.677695    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 05:51:47.705415 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:47.705383    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:48.000325 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:48.000242    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:48.181853 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:48.181803    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:48.357649 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:48.357552    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:48.581496 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:48.581463    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:49.193740 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:49.193704    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:49.353171 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:49.353117    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:49.534667 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:49.534509    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:50.135473 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:50.135387    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:51.106447 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.106417    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"]
Oct 30 05:51:51.112359 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.112338    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.120845 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.120813    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:51.136580 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.136546    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"]
Oct 30 05:51:51.250692 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.250656    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.250914 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.250713    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-data\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.250914 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.250747    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-envoy\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.250914 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.250904    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcmjx\" (UniqueName: \"kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-kube-api-access-bcmjx\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.251120 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.250960    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-certs\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.251305 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.251135    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-token\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.251370 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.251314    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-credential-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.251425 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.251394    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a0d40cd-44b7-405a-8967-7d3111e029e6-istiod-ca-cert\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.251520 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.251423    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-podinfo\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.283054 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.283023    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:51.352477 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.352445    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.352477 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.352487    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-data\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.352718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.352652    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-envoy\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.352765 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.352732    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcmjx\" (UniqueName: \"kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-kube-api-access-bcmjx\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.352820 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.352785    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-certs\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.352914 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.352897    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-token\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.352991 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.352926    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-data\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.352991 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.352901    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.353112 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.353050    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-credential-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.353112 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.353089    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-certs\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.353227 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.353184    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a0d40cd-44b7-405a-8967-7d3111e029e6-istiod-ca-cert\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.353274 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.353235    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-podinfo\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.353346 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.353323    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-credential-socket\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.354120 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.354098    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a0d40cd-44b7-405a-8967-7d3111e029e6-istiod-ca-cert\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.355399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.355380    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-envoy\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.355814 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.355793    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-podinfo\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.416796 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.416760    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-token\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.424469 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.424430    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcmjx\" (UniqueName: \"kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-kube-api-access-bcmjx\") pod \"e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") " pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.427957 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.427928    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:51.766582 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:51:51.766547    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a0d40cd_44b7_405a_8967_7d3111e029e6.slice/crio-0a7cc4bcbf9c64b2e2176fe008d5d3a0357e39c403e422e7e091ce43c9726ef8 WatchSource:0}: Error finding container 0a7cc4bcbf9c64b2e2176fe008d5d3a0357e39c403e422e7e091ce43c9726ef8: Status 404 returned error can't find the container with id 0a7cc4bcbf9c64b2e2176fe008d5d3a0357e39c403e422e7e091ce43c9726ef8
Oct 30 05:51:51.767637 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.767609    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"]
Oct 30 05:51:51.769623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.769567    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:51.769689 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.769670    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:51.769733 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:51.769711    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:51:52.124332 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:52.124247    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" event={"ID":"7a0d40cd-44b7-405a-8967-7d3111e029e6","Type":"ContainerStarted","Data":"b61695d7c06d95e5ea1e14961c897e91003db8140870c5e04f377826d69d8d19"}
Oct 30 05:51:52.124332 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:52.124299    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" event={"ID":"7a0d40cd-44b7-405a-8967-7d3111e029e6","Type":"ContainerStarted","Data":"0a7cc4bcbf9c64b2e2176fe008d5d3a0357e39c403e422e7e091ce43c9726ef8"}
Oct 30 05:51:52.185345 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:52.179336    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podStartSLOduration=1.179313595 podStartE2EDuration="1.179313595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:51:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:51:52.173441815 +0000 UTC m=+1215.918727148" watchObservedRunningTime="2026-10-30 05:51:52.179313595 +0000 UTC m=+1215.924598928"
Oct 30 05:51:52.428139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:52.428097    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:51:52.432803 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:52.432771    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:51:52.432983 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:52.432834    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:51:52.825863 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:52.825754    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:53.445613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:53.445046    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:51:53.445613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:53.445134    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:51:54.438658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:54.438623    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:51:54.438948 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:54.438919    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:51:54.504615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:54.497655    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"]
Oct 30 05:51:54.506431 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:54.506398    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:54.514663 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:54.514018    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"]
Oct 30 05:51:54.529418 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:54.529389    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-709c4831-serving-cert\""
Oct 30 05:51:54.780674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:54.752364    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:54.780674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:54.773761    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-metrics-certs\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:54.780674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:54.773973    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91eb625c-2f2b-4943-b68a-9d75753b7248-openshift-service-ca\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:54.780674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:54.774063    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:54.780674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:54.774162    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84tlt\" (UniqueName: \"kubernetes.io/projected/91eb625c-2f2b-4943-b68a-9d75753b7248-kube-api-access-84tlt\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:54.955611 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:54.951930    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:55.025683 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.011338    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91eb625c-2f2b-4943-b68a-9d75753b7248-openshift-service-ca\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:55.025683 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.011459    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:55.025683 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.011495    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84tlt\" (UniqueName: \"kubernetes.io/projected/91eb625c-2f2b-4943-b68a-9d75753b7248-kube-api-access-84tlt\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:55.032673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.026207    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91eb625c-2f2b-4943-b68a-9d75753b7248-openshift-service-ca\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:55.041617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.040782    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-metrics-certs\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:55.051827 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:55.041801    5066 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-serving-cert: secret "maas-api-e2e-ait-709c4831-serving-cert" not found
Oct 30 05:51:55.051827 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:55.041885    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls podName:91eb625c-2f2b-4943-b68a-9d75753b7248 nodeName:}" failed. No retries permitted until 2026-10-30 05:51:55.54186472 +0000 UTC m=+1219.287150046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls") pod "maas-api-e2e-ait-709c4831-57686c497f-nkbfw" (UID: "91eb625c-2f2b-4943-b68a-9d75753b7248") : secret "maas-api-e2e-ait-709c4831-serving-cert" not found
Oct 30 05:51:55.088619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.084815    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-metrics-certs\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:55.118448 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.110150    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84tlt\" (UniqueName: \"kubernetes.io/projected/91eb625c-2f2b-4943-b68a-9d75753b7248-kube-api-access-84tlt\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:55.244620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.244204    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:55.434615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.429679    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:51:55.434615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.429745    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:51:55.472617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.448166    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:55.599616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.580706    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:55.599616 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:55.580998    5066 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-serving-cert: secret "maas-api-e2e-ait-709c4831-serving-cert" not found
Oct 30 05:51:55.599616 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:55.581061    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls podName:91eb625c-2f2b-4943-b68a-9d75753b7248 nodeName:}" failed. No retries permitted until 2026-10-30 05:51:56.58104213 +0000 UTC m=+1220.326327441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls") pod "maas-api-e2e-ait-709c4831-57686c497f-nkbfw" (UID: "91eb625c-2f2b-4943-b68a-9d75753b7248") : secret "maas-api-e2e-ait-709c4831-serving-cert" not found
Oct 30 05:51:55.670629 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.669429    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:55.709673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.707690    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:55.725833 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.725781    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:55.760128 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.760088    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 05:51:55.762769 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.760367    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 05:51:55.832103 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:55.832058    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:56.207400 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.207352    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Oct 30 05:51:56.259812 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.259765    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:51:56.350536 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.350493    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:51:56.359128 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.358783    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:51:56.359530 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.359509    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:56.375613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.371686    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:56.433022 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.432984    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:51:56.433291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.433260    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:51:56.556735 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.556633    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:56.673310 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.673264    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:56.707796 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.707768    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls\") pod \"maas-api-e2e-ait-709c4831-57686c497f-nkbfw\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:56.708105 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.708076    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:56.708946 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.708923    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:51:56.728732 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.728686    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 05:51:56.729903 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.729878    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:51:56.773684 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.773645    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:56.797008 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.773933    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:56.797935 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.797305    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:51:56.798149 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.798132    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:51:56.928682 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.904232    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:51:56.928682 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:56.907272    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:51:57.234672 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.218521    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:57.538574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.277759    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:57.538574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.452092    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:51:57.538574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.452179    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:51:57.538574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.342480    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:57.538574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.427403    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 05:51:57.538574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.427494    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:57.538574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.468859    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:51:57.538574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.508716    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:51:57.538574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.512685    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:51:57.538574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.514878    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:51:57.538574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.514975    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:51:57.770275 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.770240    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 05:51:57.857670 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.842959    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": context deadline exceeded"
Oct 30 05:51:57.905634 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.858025    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:51:57.905634 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.900194    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:51:57.905634 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.860420    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:57.905634 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.859023    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 05:51:57.947488 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.921175    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:57.999218 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:57.999175    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": context deadline exceeded"
Oct 30 05:51:58.147648 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.147526    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:58.317925 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.203744    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:58.464616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.461111    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:51:58.464616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.461216    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:51:58.464616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.463867    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:58.464616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.464172    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Oct 30 05:51:58.464616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.464329    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:51:58.464616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.464489    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 05:51:58.465614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.465190    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 05:51:58.526708 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.526659    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:51:58.529697 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.529655    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:51:58.542210 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.542148    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:58.828772 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.799132    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:59.012612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:58.987562    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 05:51:59.078676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.047476    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:59.078676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.057922    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:59.371716 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.365549    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:59.469244 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.451561    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:51:59.469244 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.451640    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:51:59.492612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.473069    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Oct 30 05:51:59.492612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.474439    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:51:59.492612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.478447    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:51:59.492612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.483312    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:59.492612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.488812    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 05:51:59.552354 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.552307    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:51:59.600649 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.561300    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:51:59.627676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.602350    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/health\": context deadline exceeded"
Oct 30 05:51:59.679617 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:51:59.675850    5066 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.268s"
Oct 30 05:51:59.679617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.678903    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:51:59.679617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.678962    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:51:59.679617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.679001    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:51:59.679617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.679037    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:51:59.816665 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:51:59.805692    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:00.022381 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.016281    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 05:52:00.053927 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.053885    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:00.058314 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.058277    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:00.085676 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.085448    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:00.174652 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.164655    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"]
Oct 30 05:52:00.174652 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.164789    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:00.400771 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.384906    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 05:52:00.433145 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.433109    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:00.490658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.470722    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:52:00.490658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.470784    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:52:00.490658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.475719    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:00.490658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.476070    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Oct 30 05:52:00.493832 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.491261    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:00.520616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.494655    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:00.520616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.495044    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:00.552367 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.534827    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:52:00.552367 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.534894    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:00.635666 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.600015    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:00.635666 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.617354    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:52:00.735787 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.735747    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:52:00.786790 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.786736    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" podUID="2964d72a-feee-4fcb-8b03-5048919222cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:00.798614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.795345    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw" event={"ID":"91eb625c-2f2b-4943-b68a-9d75753b7248","Type":"ContainerStarted","Data":"270ce2119486285bcf1acd8ae83b043389653a4ec881daeebb84eae22bedcf9f"}
Oct 30 05:52:00.810793 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.810753    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:01.003617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.998045    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:52:01.003617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:00.998106    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:01.050615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.041420    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 05:52:01.072622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.056668    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:01.072622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.066674    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 05:52:01.198624 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.197669    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:01.407914 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.407874    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 05:52:01.477138 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.477089    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:01.477808 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.477773    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:52:01.493204 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.492734    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:01.502894 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.501735    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:01.502894 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.502385    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 05:52:01.696532 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.696428    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:52:01.791658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.698167    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:01.791658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.698199    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:01.791658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.764144    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:52:01.791658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.764190    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:52:01.880962 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.811220    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:02.024670 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.928328    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:52:02.026466 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.026433    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:02.026665 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.025511    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:02.026849 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:01.956465    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:02.062700 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.051119    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:02.070610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.067024    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 05:52:02.070610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.067384    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:02.319519 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.316860    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:02.416770 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.416712    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:02.437801 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.437768    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:52:02.438152 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.438125    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:52:02.486839 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.486798    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:52:02.496816 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.496759    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Oct 30 05:52:02.508209 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.508175    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 05:52:02.509800 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.508135    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:02.521238 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.521210    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:52:02.714169 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.714128    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:52:02.775615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:02.773459    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 05:52:03.071270 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.071140    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 05:52:03.094616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.075723    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:03.094616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.076143    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:03.094616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.076466    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 05:52:03.129611 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.129545    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:03.151097 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:52:03.132910    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:52:03.192655 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.133292    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:03.268117 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.268075    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/health\": context deadline exceeded"
Oct 30 05:52:03.417385 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.417339    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:03.443383 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.443346    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:52:03.443706 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.443673    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:52:03.470030 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.469993    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:03.486287 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.470421    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:52:03.511230 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.511188    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:52:03.511913 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.511882    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Oct 30 05:52:03.526243 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.526203    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:03.573486 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.573445    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:03.591608 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.591539    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw" event={"ID":"91eb625c-2f2b-4943-b68a-9d75753b7248","Type":"ContainerStarted","Data":"461f23608f3e4761d38c23ea2bd2501e205dd27f22037c0e34dae160029892de"}
Oct 30 05:52:03.592688 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.592666    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:52:03.723490 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.723394    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:03.792617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.785970    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw" podStartSLOduration=9.057383681 podStartE2EDuration="9.7859488s" totalImagesPullingTime="728.565119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:52:03.778984991 +0000 UTC m=+1227.524270317" watchObservedRunningTime="2026-10-30 05:52:03.7859488 +0000 UTC m=+1227.531234133"
Oct 30 05:52:03.831630 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.826192    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:03.831630 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.826635    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:52:03.910126 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:03.910084    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:52:04.076067 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.075965    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 05:52:04.091465 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.087751    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 05:52:04.138618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.087814    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:04.138618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.087871    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 05:52:04.347719 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.345074    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:52:04.436622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.434680    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:04.492736 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.492703    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:52:04.492957 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.492935    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:52:04.493446 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.493422    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:04.559719 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.559626    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 05:52:04.597838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.597681    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Oct 30 05:52:04.632237 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.612468    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:04.671836 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.671791    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:52:04.672033 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.671869    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:52:04.672114 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.672072    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:52:04.672177 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.672124    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:52:04.672177 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.672159    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:52:04.672283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.672192    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:52:04.672283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.672225    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:52:04.672283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.672256    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:52:04.672421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.672286    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:52:04.672421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.672316    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:52:04.672421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:04.672346    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 05:52:05.035955 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:05.035908    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:05.429245 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:05.429208    5066 patch_prober.go:29] interesting pod/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 05:52:05.429791 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:05.429269    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 05:52:05.547819 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:05.547768    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:06.428161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:06.428131    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:52:06.432555 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:06.432527    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:52:06.690628 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:06.690505    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 05:52:07.123128 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:07.123053    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:07.343269 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:07.343236    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:08.422485 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:08.422456    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:08.683834 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:08.683761    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:08.852965 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:08.852913    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:09.279473 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:09.279439    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:09.674113 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:09.674082    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:52:09.774211 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:09.774141    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:10.028778 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:10.028662    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:10.555864 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:10.555822    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:11.846099 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:11.846053    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:12.007168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:12.007119    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:12.273771 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:12.273741    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:13.997767 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:13.997737    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:14.177047 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:14.177018    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:14.382663 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:14.382537    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:15.610646 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:15.610616    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:15.817778 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:15.817723    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:16.039847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:16.039810    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:17.184764 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:17.184734    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:18.165481 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:18.165446    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:19.730840 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:19.730807    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:21.526223 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:21.526186    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:23.098720 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:23.098682    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:25.145672 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:25.145623    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:27.655751 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:27.655694    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:27.811701 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:27.810695    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 05:52:27.941253 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:27.941143    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:28.047740 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:28.047684    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:28.060707 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:28.060663    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 05:52:28.109567 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:28.108957    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:28.224673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:28.221848    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:28.496933 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:28.496825    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:28.517739 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:28.517696    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:52:28.526731 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:28.526684    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:28.569823 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:28.569786    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:28.599970 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:28.599925    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:28.621653 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:28.604222    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:28.816816 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:28.816707    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:28.817950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:28.817924    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:52:29.020625 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:29.020567    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:29.076511 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:29.076411    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:29.149615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:29.134814    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:52:29.161135 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:29.133199    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:29.162222 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:29.162195    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:52:29.223947 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:29.223392    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:29.254985 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:29.254940    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:52:29.514451 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:29.514406    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:29.514983 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:29.514960    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:52:29.537470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:29.534691    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:29.537470 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:29.536096    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:52:29.851860 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:29.840241    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:30.084194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:30.084151    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 05:52:30.088359 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:30.088334    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 05:52:30.541203 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:30.541161    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:52:30.576119 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:30.569563    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:30.596901 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:30.596853    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.65:8000/ready\": context deadline exceeded"
Oct 30 05:52:30.608845 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:30.575331    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 05:52:30.609497 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:30.569841    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:30.727621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:30.710616    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 05:52:30.804408 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:30.804315    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:30.816371 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:30.816322    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:30.949223 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:30.949166    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:31.013623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.000544    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:52:31.210363 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.210321    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:31.227730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.227690    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:31.283305 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.282762    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:31.335759 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.329983    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:31.588678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.579496    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:52:31.638938 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.594710    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 05:52:31.639304 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.628985    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": context deadline exceeded"
Oct 30 05:52:31.639304 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.628929    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:31.698381 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.698337    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:31.724445 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.724399    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 05:52:31.825824 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.825764    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:31.832035 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.831992    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:52:31.832335 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.832318    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:52:31.832541 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.832512    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:52:31.844944 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.844828    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 05:52:31.882617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.853104    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:52:31.953889 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.953839    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/health\": context deadline exceeded"
Oct 30 05:52:32.117679 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:31.999287    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:32.119116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:32.017910    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 05:52:32.231623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:32.216086    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:32.232489 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:32.232457    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:52:32.255028 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:32.233467    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 05:52:32.304829 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:32.304768    5066 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/health\": context deadline exceeded"
Oct 30 05:52:32.305243 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:32.305215    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:32.314656 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:32.306709    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:52:32.655615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:32.654927    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded"
Oct 30 05:52:32.733001 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:32.732961    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:32.863615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:32.855384    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:32.901557 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:32.901526    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 05:52:32.901934 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:32.901919    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 05:52:32.968217 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:52:32.968125    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:52:33.008329 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:33.008267    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 05:52:33.023619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:33.020932    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 05:52:33.087897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:33.078191    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 05:52:33.096623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:33.095158    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 05:52:33.096912 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:33.096882    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 05:52:33.295608 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:33.295498    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:34.865958 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:34.865922    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:35.043352 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:35.043323    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:35.209783 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:35.209745    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:36.532777 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:36.532744    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:37.170568 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:37.170532    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:37.338083 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:37.338043    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:37.504817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:37.504716    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:37.665953 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:37.665875    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:37.739612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:37.734990    5066 patch_prober.go:29] interesting pod/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:52:37.739612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:37.735058    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq" podUID="1718d33e-9c55-4afc-a897-5251337302d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:52:38.247125 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:38.247096    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:38.779735 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:38.779685    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:38.968516 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:38.968480    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"]
Oct 30 05:52:38.969115 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:38.969074    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw" podUID="91eb625c-2f2b-4943-b68a-9d75753b7248" containerName="maas-api" containerID="cri-o://461f23608f3e4761d38c23ea2bd2501e205dd27f22037c0e34dae160029892de" gracePeriod=30
Oct 30 05:52:39.032062 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.031871    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:39.205693 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.205532    5066 generic.go:356] "Generic (PLEG): container finished" podID="91eb625c-2f2b-4943-b68a-9d75753b7248" containerID="461f23608f3e4761d38c23ea2bd2501e205dd27f22037c0e34dae160029892de" exitCode=0
Oct 30 05:52:39.205693 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.205608    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw" event={"ID":"91eb625c-2f2b-4943-b68a-9d75753b7248","Type":"ContainerDied","Data":"461f23608f3e4761d38c23ea2bd2501e205dd27f22037c0e34dae160029892de"}
Oct 30 05:52:39.253197 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.253163    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:39.610008 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.601296    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:52:39.799544 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.799015    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-metrics-certs" pod "91eb625c-2f2b-4943-b68a-9d75753b7248" (UID: "91eb625c-2f2b-4943-b68a-9d75753b7248"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:52:39.824326 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.806603    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:39.824326 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.817701    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-metrics-certs\" (UniqueName: \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-metrics-certs\") pod \"91eb625c-2f2b-4943-b68a-9d75753b7248\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") "
Oct 30 05:52:39.824326 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.817789    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls\") pod \"91eb625c-2f2b-4943-b68a-9d75753b7248\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") "
Oct 30 05:52:39.824326 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.817988    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91eb625c-2f2b-4943-b68a-9d75753b7248-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91eb625c-2f2b-4943-b68a-9d75753b7248-openshift-service-ca\") pod \"91eb625c-2f2b-4943-b68a-9d75753b7248\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") "
Oct 30 05:52:39.824326 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.818026    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91eb625c-2f2b-4943-b68a-9d75753b7248-kube-api-access-84tlt\" (UniqueName: \"kubernetes.io/projected/91eb625c-2f2b-4943-b68a-9d75753b7248-kube-api-access-84tlt\") pod \"91eb625c-2f2b-4943-b68a-9d75753b7248\" (UID: \"91eb625c-2f2b-4943-b68a-9d75753b7248\") "
Oct 30 05:52:39.824326 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.823688    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91eb625c-2f2b-4943-b68a-9d75753b7248-openshift-service-ca" pod "91eb625c-2f2b-4943-b68a-9d75753b7248" (UID: "91eb625c-2f2b-4943-b68a-9d75753b7248"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:52:39.863448 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.863357    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls" pod "91eb625c-2f2b-4943-b68a-9d75753b7248" (UID: "91eb625c-2f2b-4943-b68a-9d75753b7248"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:52:39.866045 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.866015    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91eb625c-2f2b-4943-b68a-9d75753b7248-kube-api-access-84tlt" pod "91eb625c-2f2b-4943-b68a-9d75753b7248" (UID: "91eb625c-2f2b-4943-b68a-9d75753b7248"). InnerVolumeSpecName "kube-api-access-84tlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:52:39.920976 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.920932    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91eb625c-2f2b-4943-b68a-9d75753b7248-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:39.920976 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.920978    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84tlt\" (UniqueName: \"kubernetes.io/projected/91eb625c-2f2b-4943-b68a-9d75753b7248-kube-api-access-84tlt\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:39.921222 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.920996    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:39.921222 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.921011    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91eb625c-2f2b-4943-b68a-9d75753b7248-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:39.979275 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:39.979244    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:40.077648 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:40.062136    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"]
Oct 30 05:52:40.077648 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:40.064556    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq" podUID="1718d33e-9c55-4afc-a897-5251337302d9" containerName="istio-proxy" containerID="cri-o://68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5" gracePeriod=30
Oct 30 05:52:40.148953 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:40.148822    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:40.297214 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:40.297172    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw" event={"ID":"91eb625c-2f2b-4943-b68a-9d75753b7248","Type":"ContainerDied","Data":"270ce2119486285bcf1acd8ae83b043389653a4ec881daeebb84eae22bedcf9f"}
Oct 30 05:52:40.297421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:40.297251    5066 scope.go:122] "RemoveContainer" containerID="461f23608f3e4761d38c23ea2bd2501e205dd27f22037c0e34dae160029892de"
Oct 30 05:52:40.297489 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:40.297433    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"
Oct 30 05:52:40.411933 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:40.411897    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"]
Oct 30 05:52:40.411933 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:40.411938    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-709c4831-57686c497f-nkbfw"]
Oct 30 05:52:40.835435 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:40.835401    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:40.974468 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:40.974394    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:52:41.081128 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.081063    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-certs\") pod \"1718d33e-9c55-4afc-a897-5251337302d9\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") "
Oct 30 05:52:41.081316 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.081163    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1718d33e-9c55-4afc-a897-5251337302d9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1718d33e-9c55-4afc-a897-5251337302d9-istiod-ca-cert\") pod \"1718d33e-9c55-4afc-a897-5251337302d9\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") "
Oct 30 05:52:41.081316 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.081222    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-credential-socket\") pod \"1718d33e-9c55-4afc-a897-5251337302d9\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") "
Oct 30 05:52:41.081316 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.081277    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-istio-token\" (UniqueName: \"kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-istio-token\") pod \"1718d33e-9c55-4afc-a897-5251337302d9\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") "
Oct 30 05:52:41.081484 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.081354    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-socket\") pod \"1718d33e-9c55-4afc-a897-5251337302d9\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") "
Oct 30 05:52:41.081484 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.081392    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1718d33e-9c55-4afc-a897-5251337302d9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1718d33e-9c55-4afc-a897-5251337302d9-istio-podinfo\") pod \"1718d33e-9c55-4afc-a897-5251337302d9\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") "
Oct 30 05:52:41.081578 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.081518    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-data\") pod \"1718d33e-9c55-4afc-a897-5251337302d9\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") "
Oct 30 05:52:41.081690 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.081673    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-envoy\") pod \"1718d33e-9c55-4afc-a897-5251337302d9\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") "
Oct 30 05:52:41.081753 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.081729    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-kube-api-access-q5jdf\" (UniqueName: \"kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-kube-api-access-q5jdf\") pod \"1718d33e-9c55-4afc-a897-5251337302d9\" (UID: \"1718d33e-9c55-4afc-a897-5251337302d9\") "
Oct 30 05:52:41.089154 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.089040    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-socket" pod "1718d33e-9c55-4afc-a897-5251337302d9" (UID: "1718d33e-9c55-4afc-a897-5251337302d9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:52:41.089386 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.089361    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1718d33e-9c55-4afc-a897-5251337302d9-istiod-ca-cert" pod "1718d33e-9c55-4afc-a897-5251337302d9" (UID: "1718d33e-9c55-4afc-a897-5251337302d9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:52:41.090208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.089623    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-certs" pod "1718d33e-9c55-4afc-a897-5251337302d9" (UID: "1718d33e-9c55-4afc-a897-5251337302d9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:52:41.090208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.089797    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-credential-socket" pod "1718d33e-9c55-4afc-a897-5251337302d9" (UID: "1718d33e-9c55-4afc-a897-5251337302d9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:52:41.090208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.089958    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-data" pod "1718d33e-9c55-4afc-a897-5251337302d9" (UID: "1718d33e-9c55-4afc-a897-5251337302d9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:52:41.094063 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.094033    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-envoy" pod "1718d33e-9c55-4afc-a897-5251337302d9" (UID: "1718d33e-9c55-4afc-a897-5251337302d9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:52:41.094325 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.094301    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-istio-token" pod "1718d33e-9c55-4afc-a897-5251337302d9" (UID: "1718d33e-9c55-4afc-a897-5251337302d9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:52:41.097899 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.097875    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-kube-api-access-q5jdf" pod "1718d33e-9c55-4afc-a897-5251337302d9" (UID: "1718d33e-9c55-4afc-a897-5251337302d9"). InnerVolumeSpecName "kube-api-access-q5jdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:52:41.103251 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.103207    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1718d33e-9c55-4afc-a897-5251337302d9-istio-podinfo" pod "1718d33e-9c55-4afc-a897-5251337302d9" (UID: "1718d33e-9c55-4afc-a897-5251337302d9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 05:52:41.183873 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.183840    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:41.183873 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.183867    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1718d33e-9c55-4afc-a897-5251337302d9-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:41.183873 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.183878    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:41.184163 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.183886    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:41.184163 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.183894    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5jdf\" (UniqueName: \"kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-kube-api-access-q5jdf\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:41.184163 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.183905    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:41.184163 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.183914    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1718d33e-9c55-4afc-a897-5251337302d9-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:41.184163 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.183922    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1718d33e-9c55-4afc-a897-5251337302d9-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:41.184163 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.183929    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1718d33e-9c55-4afc-a897-5251337302d9-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:52:41.219398 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.219369    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"]
Oct 30 05:52:41.220185 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.220161    5066 cpu_manager.go:414] "containerMap: removing container" podUID="1718d33e-9c55-4afc-a897-5251337302d9" containerName="istio-proxy"
Oct 30 05:52:41.220185 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.220185    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1718d33e-9c55-4afc-a897-5251337302d9" containerName="istio-proxy"
Oct 30 05:52:41.220376 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.220216    5066 cpu_manager.go:414] "containerMap: removing container" podUID="91eb625c-2f2b-4943-b68a-9d75753b7248" containerName="maas-api"
Oct 30 05:52:41.220376 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.220224    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91eb625c-2f2b-4943-b68a-9d75753b7248" containerName="maas-api"
Oct 30 05:52:41.220490 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.220422    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="1718d33e-9c55-4afc-a897-5251337302d9" containerName="istio-proxy"
Oct 30 05:52:41.220490 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.220439    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="91eb625c-2f2b-4943-b68a-9d75753b7248" containerName="maas-api"
Oct 30 05:52:41.225411 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.224525    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.251934 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.249829    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e2a1565f-openshift-default-dockercfg-rbfrk\""
Oct 30 05:52:41.261217 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.261131    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"]
Oct 30 05:52:41.285301 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.285249    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dee20c96-3195-4279-ac81-09b7e125295c-istio-podinfo\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.285482 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.285331    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-envoy\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.285482 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.285452    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-credential-socket\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.285613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.285518    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-data\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.285613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.285547    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dee20c96-3195-4279-ac81-09b7e125295c-istiod-ca-cert\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.285730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.285644    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkwpf\" (UniqueName: \"kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-kube-api-access-vkwpf\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.286276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.285798    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-istio-token\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.286276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.285964    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-certs\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.286276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.286104    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-socket\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.308529 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.308497    5066 generic.go:356] "Generic (PLEG): container finished" podID="1718d33e-9c55-4afc-a897-5251337302d9" containerID="68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5" exitCode=0
Oct 30 05:52:41.308748 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.308568    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"
Oct 30 05:52:41.308748 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.308635    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq" event={"ID":"1718d33e-9c55-4afc-a897-5251337302d9","Type":"ContainerDied","Data":"68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5"}
Oct 30 05:52:41.308748 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.308699    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq" event={"ID":"1718d33e-9c55-4afc-a897-5251337302d9","Type":"ContainerDied","Data":"cf832aefd906e580170c1539d1cdc1feaa5a7d3739f417e1823f6fd5c9cc3cdf"}
Oct 30 05:52:41.308748 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.308733    5066 scope.go:122] "RemoveContainer" containerID="68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.389839    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-envoy\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.389929    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-credential-socket\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.389983    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-data\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.390010    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dee20c96-3195-4279-ac81-09b7e125295c-istiod-ca-cert\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.390048    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkwpf\" (UniqueName: \"kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-kube-api-access-vkwpf\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.390208    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-istio-token\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.390277    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-certs\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.390419    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-socket\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.390487    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dee20c96-3195-4279-ac81-09b7e125295c-istio-podinfo\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.404392    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dee20c96-3195-4279-ac81-09b7e125295c-istiod-ca-cert\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.405066    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-credential-socket\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.405825    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-data\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.406069    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-certs\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.406294    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-socket\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.436638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.428570    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:41.512805 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.495915    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-envoy\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.568004 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.551444    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dee20c96-3195-4279-ac81-09b7e125295c-istio-podinfo\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.568004 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.564112    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"]
Oct 30 05:52:41.572437 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.571704    5066 scope.go:122] "RemoveContainer" containerID="68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5"
Oct 30 05:52:41.573141 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:52:41.573000    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5\": container with ID starting with 68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5 not found: ID does not exist" containerID="68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5"
Oct 30 05:52:41.573141 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.573041    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5"} err="failed to get container status \"68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5\": rpc error: code = NotFound desc = could not find container \"68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5\": container with ID starting with 68df569f3e05e21a82b0bf0f3f1e6033951addd297b6b46db45417ad324fe2b5 not found: ID does not exist"
Oct 30 05:52:41.573964 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.573944    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-istio-token\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.575572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.575550    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkwpf\" (UniqueName: \"kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-kube-api-access-vkwpf\") pod \"e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") " pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:41.587402 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.587354    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-709c4831-openshift-default-647b55f95f-w6zlq"]
Oct 30 05:52:41.690372 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.690339    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:41.869863 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:41.869766    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:42.142743 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:42.142654    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:42.217615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:42.217561    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"]
Oct 30 05:52:42.221270 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:42.221231    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:52:42.221363 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:42.221320    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:52:42.221406 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:42.221362    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:52:42.319143 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:42.319099    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj" event={"ID":"dee20c96-3195-4279-ac81-09b7e125295c","Type":"ContainerStarted","Data":"78664860988a801a52899f23e484016e885c529b934d6fe0b20299b04dd97488"}
Oct 30 05:52:42.411041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:42.411002    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1718d33e-9c55-4afc-a897-5251337302d9" path="/var/lib/kubelet/pods/1718d33e-9c55-4afc-a897-5251337302d9/volumes"
Oct 30 05:52:42.411813 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:42.411790    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91eb625c-2f2b-4943-b68a-9d75753b7248" path="/var/lib/kubelet/pods/91eb625c-2f2b-4943-b68a-9d75753b7248/volumes"
Oct 30 05:52:42.905733 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:42.905700    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:43.222674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.222520    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:43.329502 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.329466    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj" event={"ID":"dee20c96-3195-4279-ac81-09b7e125295c","Type":"ContainerStarted","Data":"e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340"}
Oct 30 05:52:43.427868 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.427813    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj" podStartSLOduration=2.427797906 podStartE2EDuration="2.427797906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:52:43.425528148 +0000 UTC m=+1267.170813480" watchObservedRunningTime="2026-10-30 05:52:43.427797906 +0000 UTC m=+1267.173083238"
Oct 30 05:52:43.739856 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.739822    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"]
Oct 30 05:52:43.746074 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.746047    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.765253 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.765228    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 05:52:43.765253 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.765226    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 05:52:43.765417 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.765242    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 05:52:43.765452 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.765418    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7ckgr\""
Oct 30 05:52:43.780278 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.780241    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"]
Oct 30 05:52:43.845425 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.845385    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.845662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.845455    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.845662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.845501    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.845662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.845531    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.845662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.845643    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j2wt\" (UniqueName: \"kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-kube-api-access-7j2wt\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.845904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.845776    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.845904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.845868    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.870514 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.870454    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:43.876380 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.876352    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:43.947075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.947041    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.947658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.947118    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.947658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.947265    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.947658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.947303    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.947658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.947446    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.947658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.947496    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.947658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.947616    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j2wt\" (UniqueName: \"kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-kube-api-access-7j2wt\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.948536 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.948509    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.950045 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.950017    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.950150 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.950097    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.950419 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.950390    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.950830 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.950814    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.975102 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.975069    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j2wt\" (UniqueName: \"kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-kube-api-access-7j2wt\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:43.975212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:43.975104    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d54kn\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:44.061260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:44.061175    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:44.337881 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:44.337780    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:44.338252 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:44.338233    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:44.339299 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:44.339276    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:52:44.405340 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:52:44.405309    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd9d2d88_c2f7_4669_9f6f_6bdf24978b1b.slice/crio-01d7bc5313d9b62b37ae54ebe8f5a99dec5d8977229bf754a384ad959ff0541b WatchSource:0}: Error finding container 01d7bc5313d9b62b37ae54ebe8f5a99dec5d8977229bf754a384ad959ff0541b: Status 404 returned error can't find the container with id 01d7bc5313d9b62b37ae54ebe8f5a99dec5d8977229bf754a384ad959ff0541b
Oct 30 05:52:44.417891 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:44.417860    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"]
Oct 30 05:52:44.831889 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:44.831859    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:45.130736 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:45.128823    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:45.323610 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:45.323567    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:45.352836 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:45.352799    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn" event={"ID":"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b","Type":"ContainerStarted","Data":"01d7bc5313d9b62b37ae54ebe8f5a99dec5d8977229bf754a384ad959ff0541b"}
Oct 30 05:52:46.491883 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:46.491810    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:46.965885 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:46.965852    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:47.175765 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:47.175732    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:47.208618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:47.207230    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:52:47.208618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:47.207330    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:52:48.228830 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:48.228079    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:48.436545 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:48.436511    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn" event={"ID":"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b","Type":"ContainerStarted","Data":"203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c"}
Oct 30 05:52:48.436779 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:48.436723    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:48.438690 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:48.438670    5066 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-d54kn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 05:52:48.438799 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:48.438710    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn" podUID="dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 05:52:48.532723 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:48.527902    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn" podStartSLOduration=2.730563828 podStartE2EDuration="5.527882276s" totalImagesPullingTime="2.797318448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:52:48.512500455 +0000 UTC m=+1272.257785787" watchObservedRunningTime="2026-10-30 05:52:48.527882276 +0000 UTC m=+1272.273167608"
Oct 30 05:52:49.444150 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:49.444120    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 05:52:50.081939 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:50.081903    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:51.684405 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:51.684357    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:53.403906 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:53.403875    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:55.245188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:55.245155    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:57.024048 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:57.024012    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:58.580997 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:58.580963    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:58.783820 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:58.783779    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:52:59.762951 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:52:59.762918    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:00.239975 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:00.239939    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:00.449194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:00.449161    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:00.862615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:00.861702    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:01.106547 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:01.106520    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:02.309022 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:02.308986    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:02.555909 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:02.555876    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:02.559661 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:53:02.559575    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:53:04.027923 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:04.027887    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:05.141923 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:05.141892    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:05.349328 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:05.349301    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:05.913302 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:05.913243    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:06.830572 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:06.830539    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:07.034378 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:07.034348    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:07.945640 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:07.945579    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:09.834663 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:09.834624    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:11.431048 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:11.430980    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:12.994227 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:12.994190    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:14.608439 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:14.608403    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:14.824433 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:14.824398    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:16.401698 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:16.401665    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:17.925419 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:17.925370    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:19.699722 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:19.699690    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:20.295414 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:20.295378    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:20.481493 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:20.481453    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:20.648085 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:20.648000    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:21.284775 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:21.284744    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:21.590908 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:21.590827    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:23.219218 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:23.219186    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:23.408934 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:23.408899    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:23.635821 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:23.635725    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:25.336279 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:25.336242    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:27.251297 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:27.251257    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:28.664345 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:28.664311    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:29.040260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:29.040224    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:29.527706 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:29.527677    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:30.029265 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:30.029232    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:30.219611 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:30.217366    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:30.921862 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:30.921831    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:32.574579 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:53:32.574537    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:53:32.632279 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:32.632233    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:34.046019 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:34.045973    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:34.264196 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:34.264160    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:34.860870 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:34.860841    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:35.621639 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:35.621549    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:35.802865 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:35.802833    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:36.715374 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:36.715331    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:36.962244 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:36.962213    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:38.316752 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:38.316720    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:38.655985 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:38.630875    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:38.952001 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:38.951844    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:39.176464 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:39.176426    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:39.594014 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:39.593981    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:39.832028 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:39.831998    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:40.946660 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:40.946629    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:41.539439 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:41.539406    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:41.728724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:41.728539    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:41.932310 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:41.932261    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:43.108941 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:43.108903    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:43.467326 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:43.467292    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:44.078412 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:44.078378    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:44.492149 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:44.492115    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:45.028203 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:45.028171    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:45.333897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:45.333184    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"]
Oct 30 05:53:45.334413 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:45.334379    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj" podUID="dee20c96-3195-4279-ac81-09b7e125295c" containerName="istio-proxy" containerID="cri-o://e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340" gracePeriod=30
Oct 30 05:53:45.398542 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:45.398503    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:45.728854 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:45.728817    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:46.485422 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:46.485378    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:46.678263 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:46.678231    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:46.903822 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:46.903792    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:47.401311 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:47.401201    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:47.616458 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:47.616419    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:47.940241 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:47.940200    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:49.669144 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:49.669070    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:49.851472 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:49.851434    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:50.192774 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.192733    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:50.656958 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.656931    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:53:50.742691 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.742609    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-envoy\") pod \"dee20c96-3195-4279-ac81-09b7e125295c\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") "
Oct 30 05:53:50.743561 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.743540    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-data\") pod \"dee20c96-3195-4279-ac81-09b7e125295c\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") "
Oct 30 05:53:50.743904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.743874    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-data" pod "dee20c96-3195-4279-ac81-09b7e125295c" (UID: "dee20c96-3195-4279-ac81-09b7e125295c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:53:50.744039 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.744017    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-socket\") pod \"dee20c96-3195-4279-ac81-09b7e125295c\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") "
Oct 30 05:53:50.744232 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.744209    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-certs\") pod \"dee20c96-3195-4279-ac81-09b7e125295c\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") "
Oct 30 05:53:50.744334 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.744262    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-istio-token\" (UniqueName: \"kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-istio-token\") pod \"dee20c96-3195-4279-ac81-09b7e125295c\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") "
Oct 30 05:53:50.744334 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.744298    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-socket" pod "dee20c96-3195-4279-ac81-09b7e125295c" (UID: "dee20c96-3195-4279-ac81-09b7e125295c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:53:50.744450 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.744379    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dee20c96-3195-4279-ac81-09b7e125295c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dee20c96-3195-4279-ac81-09b7e125295c-istio-podinfo\") pod \"dee20c96-3195-4279-ac81-09b7e125295c\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") "
Oct 30 05:53:50.744503 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.744449    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-kube-api-access-vkwpf\" (UniqueName: \"kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-kube-api-access-vkwpf\") pod \"dee20c96-3195-4279-ac81-09b7e125295c\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") "
Oct 30 05:53:50.744557 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.744541    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-credential-socket\") pod \"dee20c96-3195-4279-ac81-09b7e125295c\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") "
Oct 30 05:53:50.744634 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.744615    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dee20c96-3195-4279-ac81-09b7e125295c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dee20c96-3195-4279-ac81-09b7e125295c-istiod-ca-cert\") pod \"dee20c96-3195-4279-ac81-09b7e125295c\" (UID: \"dee20c96-3195-4279-ac81-09b7e125295c\") "
Oct 30 05:53:50.744870 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.744844    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-certs" pod "dee20c96-3195-4279-ac81-09b7e125295c" (UID: "dee20c96-3195-4279-ac81-09b7e125295c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:53:50.745305 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.745272    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-credential-socket" pod "dee20c96-3195-4279-ac81-09b7e125295c" (UID: "dee20c96-3195-4279-ac81-09b7e125295c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:53:50.745422 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.745405    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:53:50.745494 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.745431    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:53:50.745494 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.745444    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:53:50.745494 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.745472    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:53:50.745804 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.745785    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dee20c96-3195-4279-ac81-09b7e125295c-istiod-ca-cert" pod "dee20c96-3195-4279-ac81-09b7e125295c" (UID: "dee20c96-3195-4279-ac81-09b7e125295c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:53:50.745920 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.745885    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-envoy" pod "dee20c96-3195-4279-ac81-09b7e125295c" (UID: "dee20c96-3195-4279-ac81-09b7e125295c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:53:50.747446 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.747409    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dee20c96-3195-4279-ac81-09b7e125295c-istio-podinfo" pod "dee20c96-3195-4279-ac81-09b7e125295c" (UID: "dee20c96-3195-4279-ac81-09b7e125295c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 05:53:50.748830 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.748782    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-istio-token" pod "dee20c96-3195-4279-ac81-09b7e125295c" (UID: "dee20c96-3195-4279-ac81-09b7e125295c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:53:50.750473 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.750454    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-kube-api-access-vkwpf" pod "dee20c96-3195-4279-ac81-09b7e125295c" (UID: "dee20c96-3195-4279-ac81-09b7e125295c"). InnerVolumeSpecName "kube-api-access-vkwpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:53:50.816043 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.815816    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:50.847214 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.847185    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:53:50.847214 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.847212    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dee20c96-3195-4279-ac81-09b7e125295c-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:53:50.847214 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.847222    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkwpf\" (UniqueName: \"kubernetes.io/projected/dee20c96-3195-4279-ac81-09b7e125295c-kube-api-access-vkwpf\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:53:50.847517 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.847232    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dee20c96-3195-4279-ac81-09b7e125295c-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:53:50.847517 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.847242    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dee20c96-3195-4279-ac81-09b7e125295c-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:53:50.970115 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.970085    5066 generic.go:356] "Generic (PLEG): container finished" podID="dee20c96-3195-4279-ac81-09b7e125295c" containerID="e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340" exitCode=0
Oct 30 05:53:50.970316 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.970138    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj" event={"ID":"dee20c96-3195-4279-ac81-09b7e125295c","Type":"ContainerDied","Data":"e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340"}
Oct 30 05:53:50.970316 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.970185    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj" event={"ID":"dee20c96-3195-4279-ac81-09b7e125295c","Type":"ContainerDied","Data":"78664860988a801a52899f23e484016e885c529b934d6fe0b20299b04dd97488"}
Oct 30 05:53:50.970316 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.970139    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"
Oct 30 05:53:50.970316 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.970214    5066 scope.go:122] "RemoveContainer" containerID="e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340"
Oct 30 05:53:50.982724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.982705    5066 scope.go:122] "RemoveContainer" containerID="e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340"
Oct 30 05:53:50.983014 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:53:50.982994    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340\": container with ID starting with e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340 not found: ID does not exist" containerID="e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340"
Oct 30 05:53:50.983079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:50.983022    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340"} err="failed to get container status \"e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340\": rpc error: code = NotFound desc = could not find container \"e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340\": container with ID starting with e4fb60b35801b9743f99aa69206fd414891a17f1c25ca7296450686f47fad340 not found: ID does not exist"
Oct 30 05:53:51.051720 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:51.051621    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"]
Oct 30 05:53:51.065608 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:51.065555    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e2a1565f-openshift-default-7cc9875d49-r45gj"]
Oct 30 05:53:51.310897 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:51.310810    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:51.497421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:51.497391    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:51.692042 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:51.692011    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:51.942746 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:51.942563    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:52.189862 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:52.189824    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:52.404675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:52.404650    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dee20c96-3195-4279-ac81-09b7e125295c" path="/var/lib/kubelet/pods/dee20c96-3195-4279-ac81-09b7e125295c/volumes"
Oct 30 05:53:53.877574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:53.877539    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:54.177614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:54.173000    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:56.182184 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:56.182148    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:57.319733 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.319693    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:57.754894 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.754853    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"]
Oct 30 05:53:57.755611 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.755576    5066 cpu_manager.go:414] "containerMap: removing container" podUID="dee20c96-3195-4279-ac81-09b7e125295c" containerName="istio-proxy"
Oct 30 05:53:57.755611 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.755609    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dee20c96-3195-4279-ac81-09b7e125295c" containerName="istio-proxy"
Oct 30 05:53:57.755838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.755824    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="dee20c96-3195-4279-ac81-09b7e125295c" containerName="istio-proxy"
Oct 30 05:53:57.759365 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.759344    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:57.784911 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.784877    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"]
Oct 30 05:53:57.787161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.787134    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1b2243ef-serving-cert\""
Oct 30 05:53:57.831717 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.831680    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-maas-api-tls\") pod \"maas-api-e2e-mt-1b2243ef-54bff4576-k46cx\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:57.831972 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.831952    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-metrics-certs\") pod \"maas-api-e2e-mt-1b2243ef-54bff4576-k46cx\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:57.832138 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.832113    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95858575-99c8-4f01-b9c4-46d9d3e325b3-openshift-service-ca\") pod \"maas-api-e2e-mt-1b2243ef-54bff4576-k46cx\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:57.832297 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.832278    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgvmz\" (UniqueName: \"kubernetes.io/projected/95858575-99c8-4f01-b9c4-46d9d3e325b3-kube-api-access-fgvmz\") pod \"maas-api-e2e-mt-1b2243ef-54bff4576-k46cx\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:57.904739 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.904708    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:57.934128 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.934095    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgvmz\" (UniqueName: \"kubernetes.io/projected/95858575-99c8-4f01-b9c4-46d9d3e325b3-kube-api-access-fgvmz\") pod \"maas-api-e2e-mt-1b2243ef-54bff4576-k46cx\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:57.934338 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.934147    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-maas-api-tls\") pod \"maas-api-e2e-mt-1b2243ef-54bff4576-k46cx\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:57.934338 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.934273    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-metrics-certs\") pod \"maas-api-e2e-mt-1b2243ef-54bff4576-k46cx\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:57.934467 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.934354    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95858575-99c8-4f01-b9c4-46d9d3e325b3-openshift-service-ca\") pod \"maas-api-e2e-mt-1b2243ef-54bff4576-k46cx\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:57.935049 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.934975    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95858575-99c8-4f01-b9c4-46d9d3e325b3-openshift-service-ca\") pod \"maas-api-e2e-mt-1b2243ef-54bff4576-k46cx\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:57.936998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.936974    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-metrics-certs\") pod \"maas-api-e2e-mt-1b2243ef-54bff4576-k46cx\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:57.937102 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.937007    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-maas-api-tls\") pod \"maas-api-e2e-mt-1b2243ef-54bff4576-k46cx\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:57.959888 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:57.959851    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgvmz\" (UniqueName: \"kubernetes.io/projected/95858575-99c8-4f01-b9c4-46d9d3e325b3-kube-api-access-fgvmz\") pod \"maas-api-e2e-mt-1b2243ef-54bff4576-k46cx\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:58.068410 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:58.068312    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:53:58.074454 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:58.074424    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:53:58.406391 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:53:58.406362    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95858575_99c8_4f01_b9c4_46d9d3e325b3.slice/crio-262ce46c4bee4e6a35d97f325df47d17ae0d9190e8981b5d5b3f933dd5c768d0 WatchSource:0}: Error finding container 262ce46c4bee4e6a35d97f325df47d17ae0d9190e8981b5d5b3f933dd5c768d0: Status 404 returned error can't find the container with id 262ce46c4bee4e6a35d97f325df47d17ae0d9190e8981b5d5b3f933dd5c768d0
Oct 30 05:53:58.407166 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:58.407142    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"]
Oct 30 05:53:59.060911 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:59.060872    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx" event={"ID":"95858575-99c8-4f01-b9c4-46d9d3e325b3","Type":"ContainerStarted","Data":"262ce46c4bee4e6a35d97f325df47d17ae0d9190e8981b5d5b3f933dd5c768d0"}
Oct 30 05:53:59.828811 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:53:59.828776    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:00.070629 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:00.070574    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx" event={"ID":"95858575-99c8-4f01-b9c4-46d9d3e325b3","Type":"ContainerStarted","Data":"57ec19c9b0e6cb0b7bf5e05def2387ec9606b11474808a89d5b98eef639e838c"}
Oct 30 05:54:00.070817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:00.070663    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:54:00.127128 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:00.127014    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx" podStartSLOduration=2.795000467 podStartE2EDuration="3.126996251s" totalImagesPullingTime="331.995784ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:53:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:54:00.123032879 +0000 UTC m=+1343.868318210" watchObservedRunningTime="2026-10-30 05:54:00.126996251 +0000 UTC m=+1343.872281583"
Oct 30 05:54:01.028065 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.028031    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"]
Oct 30 05:54:01.036987 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.036959    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.064949 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.064909    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"]
Oct 30 05:54:01.128507 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.128478    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-c73fcb-openshift-default-dockercfg-rtknk\""
Oct 30 05:54:01.156742 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.156708    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-credential-socket\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.156962 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.156758    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7gl4\" (UniqueName: \"kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-kube-api-access-v7gl4\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.156962 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.156783    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-certs\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.157121 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.157009    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-token\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.157187 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.157142    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istiod-ca-cert\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.157331 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.157312    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-data\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.157399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.157380    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-socket\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.157460 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.157419    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-podinfo\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.157583 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.157569    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-envoy\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.259567 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.259530    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-envoy\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.259567 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.259570    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-credential-socket\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.259809 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.259661    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gl4\" (UniqueName: \"kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-kube-api-access-v7gl4\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.259809 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.259687    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-certs\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.259809 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.259775    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-token\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.259911 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.259862    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istiod-ca-cert\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.260012 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.259988    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-data\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.260090 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.260064    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-socket\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.260144 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.260104    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-podinfo\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.260190 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.260148    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-credential-socket\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.260190 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.260176    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-certs\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.260396 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.260366    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-data\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.260525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.260434    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-socket\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.260668 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.260651    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istiod-ca-cert\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.262410 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.262387    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-envoy\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.262759 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.262743    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-podinfo\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.286493 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.286419    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7gl4\" (UniqueName: \"kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-kube-api-access-v7gl4\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.286722 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.286702    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-token\") pod \"e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") " pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.354673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.354637    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:01.607715 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.607580    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:01.688034 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:54:01.687996    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec1fd3a3_dabc_4251_8ea3_c30bf2302c6d.slice/crio-30a2b5ceae58f679a3b33c2adacc2c65e4e266d9d717f79d2d38fd3d2e1ed298 WatchSource:0}: Error finding container 30a2b5ceae58f679a3b33c2adacc2c65e4e266d9d717f79d2d38fd3d2e1ed298: Status 404 returned error can't find the container with id 30a2b5ceae58f679a3b33c2adacc2c65e4e266d9d717f79d2d38fd3d2e1ed298
Oct 30 05:54:01.692157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.692128    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"]
Oct 30 05:54:01.692283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.692127    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:54:01.692283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.692219    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:54:01.692283 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:01.692255    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:54:02.097637 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:02.097541    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x" event={"ID":"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d","Type":"ContainerStarted","Data":"6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e"}
Oct 30 05:54:02.098149 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:02.097693    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x" event={"ID":"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d","Type":"ContainerStarted","Data":"30a2b5ceae58f679a3b33c2adacc2c65e4e266d9d717f79d2d38fd3d2e1ed298"}
Oct 30 05:54:02.169716 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:02.169664    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x" podStartSLOduration=2.169643582 podStartE2EDuration="2.169643582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:54:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:54:02.165759485 +0000 UTC m=+1345.911044815" watchObservedRunningTime="2026-10-30 05:54:02.169643582 +0000 UTC m=+1345.914928917"
Oct 30 05:54:02.355569 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:02.355484    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:02.357177 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:02.357148    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 05:54:02.357301 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:02.357206    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x" podUID="ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 05:54:02.575757 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:54:02.575721    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:54:03.356171 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:03.356129    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:03.357217 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:03.357087    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 05:54:03.357434 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:03.357410    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x" podUID="ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 05:54:04.355983 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:04.355950    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 05:54:04.356176 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:04.356001    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x" podUID="ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 05:54:05.097000 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:05.096949    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:05.355574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:05.355485    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 05:54:05.355574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:05.355543    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x" podUID="ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 05:54:06.087320 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.087283    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:54:06.355384 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.355294    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 05:54:06.355384 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.355368    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x" podUID="ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 05:54:06.861754 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.861659    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"]
Oct 30 05:54:06.867761 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.867726    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"]
Oct 30 05:54:06.867933 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.867844    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:06.885040 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.885016    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w6-477f6b-serving-cert\""
Oct 30 05:54:06.897736 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.897711    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f795n\" (UniqueName: \"kubernetes.io/projected/7bb61813-598d-4817-bfd5-a3b539e195fe-kube-api-access-f795n\") pod \"maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:06.898073 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.898052    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:06.898314 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.898296    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bb61813-598d-4817-bfd5-a3b539e195fe-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:06.898494 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.898482    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:06.986740 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.986712    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:07.000008 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:06.999974    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:07.000184 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.000058    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bb61813-598d-4817-bfd5-a3b539e195fe-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:07.000184 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.000124    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:07.000305 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.000279    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f795n\" (UniqueName: \"kubernetes.io/projected/7bb61813-598d-4817-bfd5-a3b539e195fe-kube-api-access-f795n\") pod \"maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:07.000943 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.000858    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bb61813-598d-4817-bfd5-a3b539e195fe-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:07.002675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.002652    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:07.002805 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.002789    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:07.080066 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.079564    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f795n\" (UniqueName: \"kubernetes.io/projected/7bb61813-598d-4817-bfd5-a3b539e195fe-kube-api-access-f795n\") pod \"maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:07.186436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.186400    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:07.254084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.252973    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:07.366404 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.365739    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 05:54:07.366404 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.365816    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x" podUID="ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 05:54:07.554055 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.554022    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"]
Oct 30 05:54:07.555830 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:54:07.555799    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb61813_598d_4817_bfd5_a3b539e195fe.slice/crio-f16854b591a2320e36ace5656e34d5f5bea13b77e2696d163135601ca5c9b05a WatchSource:0}: Error finding container f16854b591a2320e36ace5656e34d5f5bea13b77e2696d163135601ca5c9b05a: Status 404 returned error can't find the container with id f16854b591a2320e36ace5656e34d5f5bea13b77e2696d163135601ca5c9b05a
Oct 30 05:54:07.949172 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:07.943989    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:08.175889 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:08.175850    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45" event={"ID":"7bb61813-598d-4817-bfd5-a3b539e195fe","Type":"ContainerStarted","Data":"43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1"}
Oct 30 05:54:08.176098 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:08.175933    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45" event={"ID":"7bb61813-598d-4817-bfd5-a3b539e195fe","Type":"ContainerStarted","Data":"f16854b591a2320e36ace5656e34d5f5bea13b77e2696d163135601ca5c9b05a"}
Oct 30 05:54:08.176163 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:08.176113    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:08.231099 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:08.231051    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45" podStartSLOduration=1.941027415 podStartE2EDuration="2.231036591s" totalImagesPullingTime="290.009176ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:54:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:54:08.230215682 +0000 UTC m=+1351.975501015" watchObservedRunningTime="2026-10-30 05:54:08.231036591 +0000 UTC m=+1351.976321924"
Oct 30 05:54:08.361014 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:08.360981    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:08.361523 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:08.361506    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:08.362251 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:08.362231    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 05:54:08.477678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:08.472813    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:08.694631 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:08.688793    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:09.207015 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:09.206983    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:10.816292 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:10.816247    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:12.383735 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:12.383707    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:12.544640 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:12.544583    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:12.710219 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:12.710191    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:13.011778 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:13.011697    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:13.216971 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:13.216940    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:14.189288 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:14.189261    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 05:54:14.631914 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:14.631831    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:14.872138 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:14.872091    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:16.261970 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.261938    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"]
Oct 30 05:54:16.267013 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.266993    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.289570 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.289544    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-3571f8-serving-cert\""
Oct 30 05:54:16.331049 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.331006    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-maas-api-tls\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.331256 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.331101    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mln2g\" (UniqueName: \"kubernetes.io/projected/644479bd-808e-44ff-af30-8b45b25e540c-kube-api-access-mln2g\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.331256 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.331153    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/644479bd-808e-44ff-af30-8b45b25e540c-openshift-service-ca\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.331256 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.331220    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-metrics-certs\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.336018 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.335990    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"]
Oct 30 05:54:16.432487 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.432448    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-maas-api-tls\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.432722 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.432544    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mln2g\" (UniqueName: \"kubernetes.io/projected/644479bd-808e-44ff-af30-8b45b25e540c-kube-api-access-mln2g\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.432722 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:54:16.432622    5066 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-serving-cert: secret "maas-api-e2e-api-a-3571f8-serving-cert" not found
Oct 30 05:54:16.432722 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.432639    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/644479bd-808e-44ff-af30-8b45b25e540c-openshift-service-ca\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.432722 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:54:16.432684    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-maas-api-tls podName:644479bd-808e-44ff-af30-8b45b25e540c nodeName:}" failed. No retries permitted until 2026-10-30 05:54:16.932666552 +0000 UTC m=+1360.677951863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-maas-api-tls") pod "maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk" (UID: "644479bd-808e-44ff-af30-8b45b25e540c") : secret "maas-api-e2e-api-a-3571f8-serving-cert" not found
Oct 30 05:54:16.432966 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.432743    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-metrics-certs\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.433327 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.433304    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/644479bd-808e-44ff-af30-8b45b25e540c-openshift-service-ca\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.435311 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.435292    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-metrics-certs\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.464437 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.464412    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mln2g\" (UniqueName: \"kubernetes.io/projected/644479bd-808e-44ff-af30-8b45b25e540c-kube-api-access-mln2g\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.509849 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.509796    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:16.777928 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.777899    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:16.943229 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.943198    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-maas-api-tls\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:16.946272 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:16.946243    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-maas-api-tls\") pod \"maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:17.182254 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:17.182191    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:17.535044 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:17.535012    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"]
Oct 30 05:54:17.536086 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:54:17.536058    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod644479bd_808e_44ff_af30_8b45b25e540c.slice/crio-fc7159761a462f05c151a0ae0a45baffcfe1cc87570a595179921fbcd901fdc0 WatchSource:0}: Error finding container fc7159761a462f05c151a0ae0a45baffcfe1cc87570a595179921fbcd901fdc0: Status 404 returned error can't find the container with id fc7159761a462f05c151a0ae0a45baffcfe1cc87570a595179921fbcd901fdc0
Oct 30 05:54:18.279938 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:18.279904    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk" event={"ID":"644479bd-808e-44ff-af30-8b45b25e540c","Type":"ContainerStarted","Data":"d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e"}
Oct 30 05:54:18.280099 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:18.279961    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk" event={"ID":"644479bd-808e-44ff-af30-8b45b25e540c","Type":"ContainerStarted","Data":"fc7159761a462f05c151a0ae0a45baffcfe1cc87570a595179921fbcd901fdc0"}
Oct 30 05:54:18.280099 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:18.279987    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:18.331620 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:18.331546    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk" podStartSLOduration=1.992942888 podStartE2EDuration="2.331530343s" totalImagesPullingTime="338.587455ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:54:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:54:18.328470896 +0000 UTC m=+1362.073756227" watchObservedRunningTime="2026-10-30 05:54:18.331530343 +0000 UTC m=+1362.076815678"
Oct 30 05:54:18.457317 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:18.457266    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:20.041266 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:20.041231    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:20.756574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:20.756543    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:21.688553 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:21.688474    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:23.489050 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:23.489003    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:24.298635 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:24.298565    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:54:25.943305 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:25.943268    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:27.927020 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:27.926986    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:28.806968 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:28.806937    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:29.775633 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:29.775577    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:30.146162 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:30.146038    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:31.535442 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:31.535403    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:32.554314 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:54:32.554273    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:54:33.238434 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:33.238399    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:35.125391 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:35.125353    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:36.848886 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:36.848857    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:37.122554 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:37.122473    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:38.334928 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.334888    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"]
Oct 30 05:54:38.341025 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.341004    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:38.360291 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.360258    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-d4ccc5-serving-cert\""
Oct 30 05:54:38.366623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.366579    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"]
Oct 30 05:54:38.426460 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.426425    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-maas-api-tls\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:38.426659 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.426512    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hd7d\" (UniqueName: \"kubernetes.io/projected/133f1d4d-bfc9-4517-a368-89968ec0591d-kube-api-access-9hd7d\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:38.426659 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.426650    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/133f1d4d-bfc9-4517-a368-89968ec0591d-openshift-service-ca\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:38.426764 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.426687    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-metrics-certs\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:38.486758 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.486713    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:38.532846 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.532806    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-maas-api-tls\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:38.533034 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.532893    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hd7d\" (UniqueName: \"kubernetes.io/projected/133f1d4d-bfc9-4517-a368-89968ec0591d-kube-api-access-9hd7d\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:38.533034 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.532977    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/133f1d4d-bfc9-4517-a368-89968ec0591d-openshift-service-ca\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:38.533034 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.533013    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-metrics-certs\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:38.534025 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:54:38.534000    5066 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-serving-cert: secret "maas-api-e2e-api-a-d4ccc5-serving-cert" not found
Oct 30 05:54:38.534208 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:54:38.534063    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-maas-api-tls podName:133f1d4d-bfc9-4517-a368-89968ec0591d nodeName:}" failed. No retries permitted until 2026-10-30 05:54:39.034042741 +0000 UTC m=+1382.779328064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-maas-api-tls") pod "maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv" (UID: "133f1d4d-bfc9-4517-a368-89968ec0591d") : secret "maas-api-e2e-api-a-d4ccc5-serving-cert" not found
Oct 30 05:54:38.535206 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.535184    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/133f1d4d-bfc9-4517-a368-89968ec0591d-openshift-service-ca\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:38.538250 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.538224    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-metrics-certs\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:38.563243 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.563201    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hd7d\" (UniqueName: \"kubernetes.io/projected/133f1d4d-bfc9-4517-a368-89968ec0591d-kube-api-access-9hd7d\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:38.667567 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.667529    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:38.844904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:38.844867    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:39.042935 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:39.042112    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-maas-api-tls\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:39.045685 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:39.045651    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-maas-api-tls\") pod \"maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:39.264621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:39.257371    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:39.598911 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:39.598881    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"]
Oct 30 05:54:39.599558 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:54:39.599524    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod133f1d4d_bfc9_4517_a368_89968ec0591d.slice/crio-bf0f9c531d207801b8914eb4aa4af21c9ffa01f9fc99c9f46e43af896d83732e WatchSource:0}: Error finding container bf0f9c531d207801b8914eb4aa4af21c9ffa01f9fc99c9f46e43af896d83732e: Status 404 returned error can't find the container with id bf0f9c531d207801b8914eb4aa4af21c9ffa01f9fc99c9f46e43af896d83732e
Oct 30 05:54:40.103087 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:40.103037    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:40.519847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:40.519815    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv" event={"ID":"133f1d4d-bfc9-4517-a368-89968ec0591d","Type":"ContainerStarted","Data":"96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a"}
Oct 30 05:54:40.520055 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:40.519871    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv" event={"ID":"133f1d4d-bfc9-4517-a368-89968ec0591d","Type":"ContainerStarted","Data":"bf0f9c531d207801b8914eb4aa4af21c9ffa01f9fc99c9f46e43af896d83732e"}
Oct 30 05:54:40.520055 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:40.519978    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:40.523029 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:40.523000    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:40.570723 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:40.570665    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv" podStartSLOduration=2.276808042 podStartE2EDuration="2.570646995s" totalImagesPullingTime="293.838953ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:54:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:54:40.567130954 +0000 UTC m=+1384.312416298" watchObservedRunningTime="2026-10-30 05:54:40.570646995 +0000 UTC m=+1384.315932328"
Oct 30 05:54:40.715879 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:40.715838    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:41.234861 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:41.231044    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:41.498085 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:41.498002    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:42.340187 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:42.340151    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:44.252279 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:44.252246    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:45.921488 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:45.921457    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:46.080630 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:46.080564    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:46.372345 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:46.372252    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:46.532879 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:46.532795    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:54:46.943066 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:46.943003    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:47.166232 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:47.166201    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:47.345685 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:47.341611    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:47.624578 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:47.624483    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:47.848824 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:47.848795    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:48.328887 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:48.328828    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:48.589639 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:48.589486    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:49.160448 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:49.160416    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:49.390021 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:49.389985    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:49.594602 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:49.592718    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:50.205987 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:50.205952    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:50.536857 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:50.536762    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:50.718241 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:50.718208    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:52.263580 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:52.263551    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:52.444880 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:52.444848    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:52.637388 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:52.637298    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:56.032466 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:56.032431    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:56.211862 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:56.211829    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:56.611841 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:56.611812    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:56.828468 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:56.828422    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:57.976447 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:57.976413    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:54:59.581240 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:54:59.581203    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:01.469641 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:01.464153    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:02.555126 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:55:02.555086    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:55:03.220458 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:03.220415    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:04.820882 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:04.820847    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:06.424113 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:06.424076    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:08.028145 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:08.028110    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:09.040496 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:09.040469    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:09.715674 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:09.715632    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:09.955832 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:09.955798    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:10.258625 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:10.258543    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:10.603578 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:10.603472    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:11.649512 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:11.649442    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:13.003987 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.003951    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"]
Oct 30 05:55:13.010102 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.010082    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.038070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.038038    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w6-f22a52-serving-cert\""
Oct 30 05:55:13.045462 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.045432    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"]
Oct 30 05:55:13.124644 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.120642    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.124644 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.120865    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df224375-e5f6-4a95-b645-785a72cbd3c7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.124644 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.120972    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpdhf\" (UniqueName: \"kubernetes.io/projected/df224375-e5f6-4a95-b645-785a72cbd3c7-kube-api-access-xpdhf\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.124644 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.121006    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.151836 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.151806    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:13.223000 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.222968    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.223163 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.223107    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df224375-e5f6-4a95-b645-785a72cbd3c7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.223163 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.223152    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpdhf\" (UniqueName: \"kubernetes.io/projected/df224375-e5f6-4a95-b645-785a72cbd3c7-kube-api-access-xpdhf\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.223249 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.223177    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.223840 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.223817    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df224375-e5f6-4a95-b645-785a72cbd3c7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.225626 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.225564    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.225817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.225797    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.261917 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.261424    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpdhf\" (UniqueName: \"kubernetes.io/projected/df224375-e5f6-4a95-b645-785a72cbd3c7-kube-api-access-xpdhf\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.324167 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.324131    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:13.332066 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.332040    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:13.720295 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.720258    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"]
Oct 30 05:55:13.838605 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:13.838530    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq" event={"ID":"df224375-e5f6-4a95-b645-785a72cbd3c7","Type":"ContainerStarted","Data":"700efc44f33ede01d38f36f0fc0bcbbce3542ffbd8add3bb34a997eb8ddede71"}
Oct 30 05:55:14.008814 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:14.008692    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:14.189462 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:14.187866    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:14.859442 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:14.859393    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq" event={"ID":"df224375-e5f6-4a95-b645-785a72cbd3c7","Type":"ContainerStarted","Data":"46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e"}
Oct 30 05:55:14.861438 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:14.861234    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:15.021622 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.020760    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq" podStartSLOduration=2.532314339 podStartE2EDuration="3.020740846s" totalImagesPullingTime="488.426507ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:55:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:55:15.014022679 +0000 UTC m=+1418.759308009" watchObservedRunningTime="2026-10-30 05:55:15.020740846 +0000 UTC m=+1418.766026189"
Oct 30 05:55:15.061623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.061140    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:15.264019 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.263968    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:15.451397 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.451353    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"]
Oct 30 05:55:15.456837 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.456816    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.479895 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.479861    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"]
Oct 30 05:55:15.487528 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.487409    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1b2243ef-openshift-default-dockercfg-jvrmz\""
Oct 30 05:55:15.496724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.496688    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-socket\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.496904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.496775    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-data\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.496904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.496880    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-certs\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.497012 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.496988    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw26v\" (UniqueName: \"kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-kube-api-access-bw26v\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.497071 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.497054    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/785beada-f111-44ab-a89c-e3e684cf40a5-istiod-ca-cert\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.497162 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.497140    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/785beada-f111-44ab-a89c-e3e684cf40a5-istio-podinfo\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.497251 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.497196    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-envoy\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.497313 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.497276    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-istio-token\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.497373 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.497330    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-credential-socket\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.599276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.599178    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-data\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.599276 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.599270    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-certs\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.599525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.599369    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw26v\" (UniqueName: \"kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-kube-api-access-bw26v\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.599525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.599398    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/785beada-f111-44ab-a89c-e3e684cf40a5-istiod-ca-cert\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.599525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.599481    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/785beada-f111-44ab-a89c-e3e684cf40a5-istio-podinfo\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.599777 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.599514    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-envoy\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.599777 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.599704    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-istio-token\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.599895 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.599782    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-credential-socket\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.599895 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.599881    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-socket\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.600398 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.600366    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-socket\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.600694 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.600672    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-data\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.600926 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.600907    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-certs\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.602274 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.602253    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-credential-socket\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.602968 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.602924    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/785beada-f111-44ab-a89c-e3e684cf40a5-istiod-ca-cert\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.605120 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.605088    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-envoy\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.605530 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.605507    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/785beada-f111-44ab-a89c-e3e684cf40a5-istio-podinfo\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.631739 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.631707    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw26v\" (UniqueName: \"kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-kube-api-access-bw26v\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.632122 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.632100    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-istio-token\") pod \"e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") " pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:15.704124 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.704066    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:15.774784 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:15.774749    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:16.131307 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:16.131277    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"]
Oct 30 05:55:16.133583 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:55:16.133557    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785beada_f111_44ab_a89c_e3e684cf40a5.slice/crio-1e72f44bbb0081dfbf732ae5cfbcc9db7e0b440ddc80d633460ed8092f205544 WatchSource:0}: Error finding container 1e72f44bbb0081dfbf732ae5cfbcc9db7e0b440ddc80d633460ed8092f205544: Status 404 returned error can't find the container with id 1e72f44bbb0081dfbf732ae5cfbcc9db7e0b440ddc80d633460ed8092f205544
Oct 30 05:55:16.137020 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:16.136983    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:55:16.137125 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:16.137076    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:55:16.137194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:16.137128    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:55:16.876849 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:16.876811    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz" event={"ID":"785beada-f111-44ab-a89c-e3e684cf40a5","Type":"ContainerStarted","Data":"33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7"}
Oct 30 05:55:16.877038 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:16.876881    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz" event={"ID":"785beada-f111-44ab-a89c-e3e684cf40a5","Type":"ContainerStarted","Data":"1e72f44bbb0081dfbf732ae5cfbcc9db7e0b440ddc80d633460ed8092f205544"}
Oct 30 05:55:16.960987 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:16.960922    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz" podStartSLOduration=1.960901397 podStartE2EDuration="1.960901397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:55:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:55:16.944051954 +0000 UTC m=+1420.689337287" watchObservedRunningTime="2026-10-30 05:55:16.960901397 +0000 UTC m=+1420.706186731"
Oct 30 05:55:17.187704 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:17.187670    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:17.472664 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:17.472233    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:17.775952 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:17.775862    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:17.779803 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:17.778730    5066 patch_prober.go:29] interesting pod/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 05:55:17.779803 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:17.778779    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz" podUID="785beada-f111-44ab-a89c-e3e684cf40a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 05:55:18.284473 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:18.284436    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:18.776383 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:18.776342    5066 patch_prober.go:29] interesting pod/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 05:55:18.776563 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:18.776427    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz" podUID="785beada-f111-44ab-a89c-e3e684cf40a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 05:55:19.257941 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:19.256405    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:19.456966 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:19.456932    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:19.805612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:19.793014    5066 patch_prober.go:29] interesting pod/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 05:55:19.805612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:19.793075    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz" podUID="785beada-f111-44ab-a89c-e3e684cf40a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 05:55:20.200621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.200568    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"]
Oct 30 05:55:20.206411 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.206391    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.259683 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.259652    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"]
Oct 30 05:55:20.268908 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.268874    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-cc33d3-serving-cert\""
Oct 30 05:55:20.303653 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.303580    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.303810 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.303733    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.303886 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.303870    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3807bc0c-f824-412d-872a-97d1d5e3b1fd-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.303920 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.303904    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5hkk\" (UniqueName: \"kubernetes.io/projected/3807bc0c-f824-412d-872a-97d1d5e3b1fd-kube-api-access-s5hkk\") pod \"maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.385040 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.385001    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:20.406048 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.406013    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.406256 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.406093    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.406351 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.406329    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3807bc0c-f824-412d-872a-97d1d5e3b1fd-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.406405 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.406385    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5hkk\" (UniqueName: \"kubernetes.io/projected/3807bc0c-f824-412d-872a-97d1d5e3b1fd-kube-api-access-s5hkk\") pod \"maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.407158 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.407107    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3807bc0c-f824-412d-872a-97d1d5e3b1fd-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.411497 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.410050    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.411497 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.410913    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.455726 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.448313    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5hkk\" (UniqueName: \"kubernetes.io/projected/3807bc0c-f824-412d-872a-97d1d5e3b1fd-kube-api-access-s5hkk\") pod \"maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.551615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.550974    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:20.556313 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.556282    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:20.776323 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.776223    5066 patch_prober.go:29] interesting pod/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 05:55:20.776323 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.776279    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz" podUID="785beada-f111-44ab-a89c-e3e684cf40a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 05:55:20.915189 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:55:20.915107    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3807bc0c_f824_412d_872a_97d1d5e3b1fd.slice/crio-6146605a494b344555d5b4f694463d25efcd374d5c33e7511aed9063f2e3014e WatchSource:0}: Error finding container 6146605a494b344555d5b4f694463d25efcd374d5c33e7511aed9063f2e3014e: Status 404 returned error can't find the container with id 6146605a494b344555d5b4f694463d25efcd374d5c33e7511aed9063f2e3014e
Oct 30 05:55:20.918354 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:20.918330    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"]
Oct 30 05:55:21.477880 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:21.473075    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:21.645208 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:21.645048    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:21.787785 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:21.787751    5066 patch_prober.go:29] interesting pod/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 05:55:21.840540 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:21.829703    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz" podUID="785beada-f111-44ab-a89c-e3e684cf40a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 05:55:21.943482 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:21.943348    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:21.991874 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:21.991844    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:21.993838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:21.993808    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm" event={"ID":"3807bc0c-f824-412d-872a-97d1d5e3b1fd","Type":"ContainerStarted","Data":"076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045"}
Oct 30 05:55:21.994004 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:21.993863    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm" event={"ID":"3807bc0c-f824-412d-872a-97d1d5e3b1fd","Type":"ContainerStarted","Data":"6146605a494b344555d5b4f694463d25efcd374d5c33e7511aed9063f2e3014e"}
Oct 30 05:55:21.994582 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:21.994561    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:22.103372 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:22.103296    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm" podStartSLOduration=1.712304462 podStartE2EDuration="2.10327776s" totalImagesPullingTime="390.973298ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:55:22.099435216 +0000 UTC m=+1425.844720547" watchObservedRunningTime="2026-10-30 05:55:22.10327776 +0000 UTC m=+1425.848563094"
Oct 30 05:55:22.746613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:22.746409    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:22.776636 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:22.776578    5066 patch_prober.go:29] interesting pod/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Oct 30 05:55:22.776814 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:22.776689    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz" podUID="785beada-f111-44ab-a89c-e3e684cf40a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Oct 30 05:55:23.252631 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:23.250428    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:23.507977 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:23.507879    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:23.781519 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:23.781423    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:23.783106 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:23.783084    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:23.784038 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:23.784016    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:24.512096 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:24.512061    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:26.253079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:26.253047    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:26.432211 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:26.432175    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:27.793949 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:27.793907    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:27.992098 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:27.992042    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:28.334078 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:28.334038    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:28.943918 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:28.943882    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:29.040368 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:29.040343    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:29.563813 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:29.563777    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:29.850566 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:29.850477    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:30.242308 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:30.242278    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:31.807355 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:31.806401    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:32.553640 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:55:32.553571    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:55:33.658144 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:33.658107    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:35.157211 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:35.157165    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:36.652106 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:36.652074    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:38.356398 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:38.356364    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:39.855912 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:39.855873    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:41.499212 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:41.499180    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:43.243050 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:43.243009    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:44.396189 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.396152    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"]
Oct 30 05:55:44.396632 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.396381    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq" podUID="df224375-e5f6-4a95-b645-785a72cbd3c7" containerName="maas-api" containerID="cri-o://46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e" gracePeriod=30
Oct 30 05:55:44.452203 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.452168    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"]
Oct 30 05:55:44.452388 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.452245    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"]
Oct 30 05:55:44.452519 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.452493    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk" podUID="644479bd-808e-44ff-af30-8b45b25e540c" containerName="maas-api" containerID="cri-o://d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e" gracePeriod=30
Oct 30 05:55:44.452627 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.452526    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv" podUID="133f1d4d-bfc9-4517-a368-89968ec0591d" containerName="maas-api" containerID="cri-o://96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a" gracePeriod=30
Oct 30 05:55:44.465420 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.463658    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"]
Oct 30 05:55:44.465420 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.463930    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz" podUID="785beada-f111-44ab-a89c-e3e684cf40a5" containerName="istio-proxy" containerID="cri-o://33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7" gracePeriod=30
Oct 30 05:55:44.532636 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.523552    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"]
Oct 30 05:55:44.532636 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.523795    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm" podUID="3807bc0c-f824-412d-872a-97d1d5e3b1fd" containerName="maas-api" containerID="cri-o://076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045" gracePeriod=30
Oct 30 05:55:44.828925 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.828893    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:55:44.845259 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.844977    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:44.923687 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.915729    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df224375-e5f6-4a95-b645-785a72cbd3c7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df224375-e5f6-4a95-b645-785a72cbd3c7-openshift-service-ca\") pod \"df224375-e5f6-4a95-b645-785a72cbd3c7\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") "
Oct 30 05:55:44.923687 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.915804    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/644479bd-808e-44ff-af30-8b45b25e540c-kube-api-access-mln2g\" (UniqueName: \"kubernetes.io/projected/644479bd-808e-44ff-af30-8b45b25e540c-kube-api-access-mln2g\") pod \"644479bd-808e-44ff-af30-8b45b25e540c\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") "
Oct 30 05:55:44.923687 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.915938    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df224375-e5f6-4a95-b645-785a72cbd3c7-kube-api-access-xpdhf\" (UniqueName: \"kubernetes.io/projected/df224375-e5f6-4a95-b645-785a72cbd3c7-kube-api-access-xpdhf\") pod \"df224375-e5f6-4a95-b645-785a72cbd3c7\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") "
Oct 30 05:55:44.923687 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.915973    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-maas-api-tls\") pod \"644479bd-808e-44ff-af30-8b45b25e540c\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") "
Oct 30 05:55:44.923687 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.916145    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-metrics-certs\") pod \"644479bd-808e-44ff-af30-8b45b25e540c\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") "
Oct 30 05:55:44.923687 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.916202    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-maas-api-tls\") pod \"df224375-e5f6-4a95-b645-785a72cbd3c7\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") "
Oct 30 05:55:44.923687 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.916436    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-metrics-certs\") pod \"df224375-e5f6-4a95-b645-785a72cbd3c7\" (UID: \"df224375-e5f6-4a95-b645-785a72cbd3c7\") "
Oct 30 05:55:44.923687 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.916580    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/644479bd-808e-44ff-af30-8b45b25e540c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/644479bd-808e-44ff-af30-8b45b25e540c-openshift-service-ca\") pod \"644479bd-808e-44ff-af30-8b45b25e540c\" (UID: \"644479bd-808e-44ff-af30-8b45b25e540c\") "
Oct 30 05:55:44.925220 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.924831    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/644479bd-808e-44ff-af30-8b45b25e540c-openshift-service-ca" pod "644479bd-808e-44ff-af30-8b45b25e540c" (UID: "644479bd-808e-44ff-af30-8b45b25e540c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:55:44.929015 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.928727    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df224375-e5f6-4a95-b645-785a72cbd3c7-openshift-service-ca" pod "df224375-e5f6-4a95-b645-785a72cbd3c7" (UID: "df224375-e5f6-4a95-b645-785a72cbd3c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:55:44.956183 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.946898    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-metrics-certs" pod "df224375-e5f6-4a95-b645-785a72cbd3c7" (UID: "df224375-e5f6-4a95-b645-785a72cbd3c7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:55:44.956493 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.952544    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/644479bd-808e-44ff-af30-8b45b25e540c-kube-api-access-mln2g" pod "644479bd-808e-44ff-af30-8b45b25e540c" (UID: "644479bd-808e-44ff-af30-8b45b25e540c"). InnerVolumeSpecName "kube-api-access-mln2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:55:44.956925 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.949168    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:44.956925 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.956886    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-metrics-certs" pod "644479bd-808e-44ff-af30-8b45b25e540c" (UID: "644479bd-808e-44ff-af30-8b45b25e540c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:55:44.957125 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.952991    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-maas-api-tls" pod "df224375-e5f6-4a95-b645-785a72cbd3c7" (UID: "df224375-e5f6-4a95-b645-785a72cbd3c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:55:44.957125 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.955241    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df224375-e5f6-4a95-b645-785a72cbd3c7-kube-api-access-xpdhf" pod "df224375-e5f6-4a95-b645-785a72cbd3c7" (UID: "df224375-e5f6-4a95-b645-785a72cbd3c7"). InnerVolumeSpecName "kube-api-access-xpdhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:55:44.959642 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.959553    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-maas-api-tls" pod "644479bd-808e-44ff-af30-8b45b25e540c" (UID: "644479bd-808e-44ff-af30-8b45b25e540c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:55:44.997089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:44.997061    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:45.033333 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.033305    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-maas-api-tls\") pod \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") "
Oct 30 05:55:45.033634 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.033616    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3807bc0c-f824-412d-872a-97d1d5e3b1fd-kube-api-access-s5hkk\" (UniqueName: \"kubernetes.io/projected/3807bc0c-f824-412d-872a-97d1d5e3b1fd-kube-api-access-s5hkk\") pod \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") "
Oct 30 05:55:45.033838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.033824    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3807bc0c-f824-412d-872a-97d1d5e3b1fd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3807bc0c-f824-412d-872a-97d1d5e3b1fd-openshift-service-ca\") pod \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") "
Oct 30 05:55:45.034106 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.034093    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-metrics-certs\") pod \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\" (UID: \"3807bc0c-f824-412d-872a-97d1d5e3b1fd\") "
Oct 30 05:55:45.034896 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.034873    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.035031 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.035019    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/644479bd-808e-44ff-af30-8b45b25e540c-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.035111 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.035100    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df224375-e5f6-4a95-b645-785a72cbd3c7-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.035189 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.035179    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mln2g\" (UniqueName: \"kubernetes.io/projected/644479bd-808e-44ff-af30-8b45b25e540c-kube-api-access-mln2g\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.035275 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.035265    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpdhf\" (UniqueName: \"kubernetes.io/projected/df224375-e5f6-4a95-b645-785a72cbd3c7-kube-api-access-xpdhf\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.035362 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.035350    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.035482 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.035463    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/644479bd-808e-44ff-af30-8b45b25e540c-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.035550 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.035488    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df224375-e5f6-4a95-b645-785a72cbd3c7-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.035550 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.035228    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3807bc0c-f824-412d-872a-97d1d5e3b1fd-openshift-service-ca" pod "3807bc0c-f824-412d-872a-97d1d5e3b1fd" (UID: "3807bc0c-f824-412d-872a-97d1d5e3b1fd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:55:45.038122 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.038074    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-maas-api-tls" pod "3807bc0c-f824-412d-872a-97d1d5e3b1fd" (UID: "3807bc0c-f824-412d-872a-97d1d5e3b1fd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:55:45.038227 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.038108    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-metrics-certs" pod "3807bc0c-f824-412d-872a-97d1d5e3b1fd" (UID: "3807bc0c-f824-412d-872a-97d1d5e3b1fd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:55:45.038412 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.038365    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3807bc0c-f824-412d-872a-97d1d5e3b1fd-kube-api-access-s5hkk" pod "3807bc0c-f824-412d-872a-97d1d5e3b1fd" (UID: "3807bc0c-f824-412d-872a-97d1d5e3b1fd"). InnerVolumeSpecName "kube-api-access-s5hkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:55:45.041038 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.041014    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:55:45.137230 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.137201    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/133f1d4d-bfc9-4517-a368-89968ec0591d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/133f1d4d-bfc9-4517-a368-89968ec0591d-openshift-service-ca\") pod \"133f1d4d-bfc9-4517-a368-89968ec0591d\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") "
Oct 30 05:55:45.137817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.137795    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/133f1d4d-bfc9-4517-a368-89968ec0591d-kube-api-access-9hd7d\" (UniqueName: \"kubernetes.io/projected/133f1d4d-bfc9-4517-a368-89968ec0591d-kube-api-access-9hd7d\") pod \"133f1d4d-bfc9-4517-a368-89968ec0591d\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") "
Oct 30 05:55:45.137994 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.137978    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-maas-api-tls\") pod \"133f1d4d-bfc9-4517-a368-89968ec0591d\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") "
Oct 30 05:55:45.138143 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.138130    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-metrics-certs\") pod \"133f1d4d-bfc9-4517-a368-89968ec0591d\" (UID: \"133f1d4d-bfc9-4517-a368-89968ec0591d\") "
Oct 30 05:55:45.138953 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.138932    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.139465 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.139450    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3807bc0c-f824-412d-872a-97d1d5e3b1fd-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.139578 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.139567    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5hkk\" (UniqueName: \"kubernetes.io/projected/3807bc0c-f824-412d-872a-97d1d5e3b1fd-kube-api-access-s5hkk\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.139696 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.139684    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3807bc0c-f824-412d-872a-97d1d5e3b1fd-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.140422 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.140400    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/133f1d4d-bfc9-4517-a368-89968ec0591d-openshift-service-ca" pod "133f1d4d-bfc9-4517-a368-89968ec0591d" (UID: "133f1d4d-bfc9-4517-a368-89968ec0591d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:55:45.175060 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.174969    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-metrics-certs" pod "133f1d4d-bfc9-4517-a368-89968ec0591d" (UID: "133f1d4d-bfc9-4517-a368-89968ec0591d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:55:45.175060 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.175001    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/133f1d4d-bfc9-4517-a368-89968ec0591d-kube-api-access-9hd7d" pod "133f1d4d-bfc9-4517-a368-89968ec0591d" (UID: "133f1d4d-bfc9-4517-a368-89968ec0591d"). InnerVolumeSpecName "kube-api-access-9hd7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:55:45.177423 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.177389    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-maas-api-tls" pod "133f1d4d-bfc9-4517-a368-89968ec0591d" (UID: "133f1d4d-bfc9-4517-a368-89968ec0591d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:55:45.215217 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.215189    5066 generic.go:356] "Generic (PLEG): container finished" podID="df224375-e5f6-4a95-b645-785a72cbd3c7" containerID="46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e" exitCode=0
Oct 30 05:55:45.215399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.215248    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"
Oct 30 05:55:45.215399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.215270    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq" event={"ID":"df224375-e5f6-4a95-b645-785a72cbd3c7","Type":"ContainerDied","Data":"46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e"}
Oct 30 05:55:45.215399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.215330    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq" event={"ID":"df224375-e5f6-4a95-b645-785a72cbd3c7","Type":"ContainerDied","Data":"700efc44f33ede01d38f36f0fc0bcbbce3542ffbd8add3bb34a997eb8ddede71"}
Oct 30 05:55:45.215399 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.215358    5066 scope.go:122] "RemoveContainer" containerID="46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e"
Oct 30 05:55:45.216823 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.216801    5066 generic.go:356] "Generic (PLEG): container finished" podID="133f1d4d-bfc9-4517-a368-89968ec0591d" containerID="96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a" exitCode=0
Oct 30 05:55:45.216959 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.216922    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv" event={"ID":"133f1d4d-bfc9-4517-a368-89968ec0591d","Type":"ContainerDied","Data":"96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a"}
Oct 30 05:55:45.216959 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.216954    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv" event={"ID":"133f1d4d-bfc9-4517-a368-89968ec0591d","Type":"ContainerDied","Data":"bf0f9c531d207801b8914eb4aa4af21c9ffa01f9fc99c9f46e43af896d83732e"}
Oct 30 05:55:45.217112 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.217024    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"
Oct 30 05:55:45.221517 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.218608    5066 generic.go:356] "Generic (PLEG): container finished" podID="644479bd-808e-44ff-af30-8b45b25e540c" containerID="d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e" exitCode=0
Oct 30 05:55:45.221517 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.218746    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk" event={"ID":"644479bd-808e-44ff-af30-8b45b25e540c","Type":"ContainerDied","Data":"d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e"}
Oct 30 05:55:45.221517 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.218789    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk" event={"ID":"644479bd-808e-44ff-af30-8b45b25e540c","Type":"ContainerDied","Data":"fc7159761a462f05c151a0ae0a45baffcfe1cc87570a595179921fbcd901fdc0"}
Oct 30 05:55:45.221517 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.218857    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"
Oct 30 05:55:45.233875 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.229036    5066 generic.go:356] "Generic (PLEG): container finished" podID="3807bc0c-f824-412d-872a-97d1d5e3b1fd" containerID="076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045" exitCode=0
Oct 30 05:55:45.233875 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.229145    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm" event={"ID":"3807bc0c-f824-412d-872a-97d1d5e3b1fd","Type":"ContainerDied","Data":"076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045"}
Oct 30 05:55:45.233875 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.229190    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm" event={"ID":"3807bc0c-f824-412d-872a-97d1d5e3b1fd","Type":"ContainerDied","Data":"6146605a494b344555d5b4f694463d25efcd374d5c33e7511aed9063f2e3014e"}
Oct 30 05:55:45.233875 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.229268    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"
Oct 30 05:55:45.234300 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.234277    5066 scope.go:122] "RemoveContainer" containerID="46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e"
Oct 30 05:55:45.234699 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:55:45.234674    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e\": container with ID starting with 46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e not found: ID does not exist" containerID="46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e"
Oct 30 05:55:45.234831 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.234712    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e"} err="failed to get container status \"46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e\": rpc error: code = NotFound desc = could not find container \"46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e\": container with ID starting with 46682cea06d7f4b57cb2574ddc253aa90380f61495b89f0bb415f5ec42ab211e not found: ID does not exist"
Oct 30 05:55:45.234831 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.234738    5066 scope.go:122] "RemoveContainer" containerID="96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a"
Oct 30 05:55:45.243714 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.241289    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hd7d\" (UniqueName: \"kubernetes.io/projected/133f1d4d-bfc9-4517-a368-89968ec0591d-kube-api-access-9hd7d\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.243714 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.241315    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.243714 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.241330    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/133f1d4d-bfc9-4517-a368-89968ec0591d-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.243714 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.241343    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/133f1d4d-bfc9-4517-a368-89968ec0591d-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:45.247646 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.247629    5066 scope.go:122] "RemoveContainer" containerID="96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a"
Oct 30 05:55:45.247942 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:55:45.247921    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a\": container with ID starting with 96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a not found: ID does not exist" containerID="96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a"
Oct 30 05:55:45.248024 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.247955    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a"} err="failed to get container status \"96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a\": rpc error: code = NotFound desc = could not find container \"96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a\": container with ID starting with 96053480585ca5ef74f734d528be59712e9a8a548f75e3e6afcc11ffcdea335a not found: ID does not exist"
Oct 30 05:55:45.248024 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.247979    5066 scope.go:122] "RemoveContainer" containerID="d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e"
Oct 30 05:55:45.271078 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.271058    5066 scope.go:122] "RemoveContainer" containerID="d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e"
Oct 30 05:55:45.271736 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:55:45.271704    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e\": container with ID starting with d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e not found: ID does not exist" containerID="d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e"
Oct 30 05:55:45.272090 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.272059    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e"} err="failed to get container status \"d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e\": rpc error: code = NotFound desc = could not find container \"d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e\": container with ID starting with d45e987a512b7c6d6b2b9059c4201cc32b83e65678ab6257592201c93d51128e not found: ID does not exist"
Oct 30 05:55:45.272303 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.272251    5066 scope.go:122] "RemoveContainer" containerID="076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045"
Oct 30 05:55:45.282684 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.282647    5066 scope.go:122] "RemoveContainer" containerID="076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045"
Oct 30 05:55:45.282974 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:55:45.282954    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045\": container with ID starting with 076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045 not found: ID does not exist" containerID="076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045"
Oct 30 05:55:45.283046 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.282984    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045"} err="failed to get container status \"076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045\": rpc error: code = NotFound desc = could not find container \"076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045\": container with ID starting with 076c857e93f877f99486729fe75bcba5f32e50765e7df3f4d75e98d4f8c44045 not found: ID does not exist"
Oct 30 05:55:45.358505 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.358470    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"]
Oct 30 05:55:45.396215 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.396182    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3571f8-d6b59cd5d-d97rk"]
Oct 30 05:55:45.478346 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.478256    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"]
Oct 30 05:55:45.489005 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.488974    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-cc33d3-69f7cfc5df-sgxrm"]
Oct 30 05:55:45.525717 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.525690    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"]
Oct 30 05:55:45.529282 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.529262    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-frdtq"]
Oct 30 05:55:45.575375 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.575346    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"]
Oct 30 05:55:45.578936 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.578913    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d4ccc5-dd9ff4874-csthv"]
Oct 30 05:55:45.762205 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:45.762130    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:46.403974 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:46.403931    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="133f1d4d-bfc9-4517-a368-89968ec0591d" path="/var/lib/kubelet/pods/133f1d4d-bfc9-4517-a368-89968ec0591d/volumes"
Oct 30 05:55:46.404709 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:46.404687    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3807bc0c-f824-412d-872a-97d1d5e3b1fd" path="/var/lib/kubelet/pods/3807bc0c-f824-412d-872a-97d1d5e3b1fd/volumes"
Oct 30 05:55:46.405260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:46.405239    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="644479bd-808e-44ff-af30-8b45b25e540c" path="/var/lib/kubelet/pods/644479bd-808e-44ff-af30-8b45b25e540c/volumes"
Oct 30 05:55:46.406145 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:46.406130    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df224375-e5f6-4a95-b645-785a72cbd3c7" path="/var/lib/kubelet/pods/df224375-e5f6-4a95-b645-785a72cbd3c7/volumes"
Oct 30 05:55:46.789243 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:46.789165    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:48.495513 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:48.495482    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:48.657946 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:48.657912    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:48.836537 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:48.836444    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:49.815112 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.815088    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:49.919249 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.919212    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"]
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.938427    5066 cpu_manager.go:414] "containerMap: removing container" podUID="644479bd-808e-44ff-af30-8b45b25e540c" containerName="maas-api"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.938467    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="644479bd-808e-44ff-af30-8b45b25e540c" containerName="maas-api"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.938561    5066 cpu_manager.go:414] "containerMap: removing container" podUID="133f1d4d-bfc9-4517-a368-89968ec0591d" containerName="maas-api"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.938570    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="133f1d4d-bfc9-4517-a368-89968ec0591d" containerName="maas-api"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.938599    5066 cpu_manager.go:414] "containerMap: removing container" podUID="785beada-f111-44ab-a89c-e3e684cf40a5" containerName="istio-proxy"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.938608    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="785beada-f111-44ab-a89c-e3e684cf40a5" containerName="istio-proxy"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.938626    5066 cpu_manager.go:414] "containerMap: removing container" podUID="3807bc0c-f824-412d-872a-97d1d5e3b1fd" containerName="maas-api"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.938632    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3807bc0c-f824-412d-872a-97d1d5e3b1fd" containerName="maas-api"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.938664    5066 cpu_manager.go:414] "containerMap: removing container" podUID="df224375-e5f6-4a95-b645-785a72cbd3c7" containerName="maas-api"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.938674    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df224375-e5f6-4a95-b645-785a72cbd3c7" containerName="maas-api"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.939021    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="644479bd-808e-44ff-af30-8b45b25e540c" containerName="maas-api"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.939066    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="133f1d4d-bfc9-4517-a368-89968ec0591d" containerName="maas-api"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.939078    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="785beada-f111-44ab-a89c-e3e684cf40a5" containerName="istio-proxy"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.939088    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="3807bc0c-f824-412d-872a-97d1d5e3b1fd" containerName="maas-api"
Oct 30 05:55:49.939089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.939104    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="df224375-e5f6-4a95-b645-785a72cbd3c7" containerName="maas-api"
Oct 30 05:55:49.960075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.923672    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-kube-api-access-bw26v\" (UniqueName: \"kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-kube-api-access-bw26v\") pod \"785beada-f111-44ab-a89c-e3e684cf40a5\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") "
Oct 30 05:55:49.960075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.956718    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-credential-socket\") pod \"785beada-f111-44ab-a89c-e3e684cf40a5\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") "
Oct 30 05:55:49.960075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.956991    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-istio-token\" (UniqueName: \"kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-istio-token\") pod \"785beada-f111-44ab-a89c-e3e684cf40a5\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") "
Oct 30 05:55:49.960075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.957202    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/785beada-f111-44ab-a89c-e3e684cf40a5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/785beada-f111-44ab-a89c-e3e684cf40a5-istio-podinfo\") pod \"785beada-f111-44ab-a89c-e3e684cf40a5\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") "
Oct 30 05:55:49.960075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.957346    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-envoy\") pod \"785beada-f111-44ab-a89c-e3e684cf40a5\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") "
Oct 30 05:55:49.960075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.957453    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-socket\") pod \"785beada-f111-44ab-a89c-e3e684cf40a5\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") "
Oct 30 05:55:49.960075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.957604    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-data\") pod \"785beada-f111-44ab-a89c-e3e684cf40a5\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") "
Oct 30 05:55:49.960075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.957655    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/785beada-f111-44ab-a89c-e3e684cf40a5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/785beada-f111-44ab-a89c-e3e684cf40a5-istiod-ca-cert\") pod \"785beada-f111-44ab-a89c-e3e684cf40a5\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") "
Oct 30 05:55:49.960075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.957789    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-certs\") pod \"785beada-f111-44ab-a89c-e3e684cf40a5\" (UID: \"785beada-f111-44ab-a89c-e3e684cf40a5\") "
Oct 30 05:55:49.987621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.979656    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-data" pod "785beada-f111-44ab-a89c-e3e684cf40a5" (UID: "785beada-f111-44ab-a89c-e3e684cf40a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:55:49.987621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.980022    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-socket" pod "785beada-f111-44ab-a89c-e3e684cf40a5" (UID: "785beada-f111-44ab-a89c-e3e684cf40a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:55:49.987621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.980189    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-credential-socket" pod "785beada-f111-44ab-a89c-e3e684cf40a5" (UID: "785beada-f111-44ab-a89c-e3e684cf40a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:55:49.987621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.980450    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/785beada-f111-44ab-a89c-e3e684cf40a5-istiod-ca-cert" pod "785beada-f111-44ab-a89c-e3e684cf40a5" (UID: "785beada-f111-44ab-a89c-e3e684cf40a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:55:49.987621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.984907    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-certs" pod "785beada-f111-44ab-a89c-e3e684cf40a5" (UID: "785beada-f111-44ab-a89c-e3e684cf40a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:55:49.987621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.985332    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:49.987621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.985354    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/785beada-f111-44ab-a89c-e3e684cf40a5-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:49.987621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.985366    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:49.987621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.985384    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:49.987621 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.985396    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:50.003728 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.994064    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/785beada-f111-44ab-a89c-e3e684cf40a5-istio-podinfo" pod "785beada-f111-44ab-a89c-e3e684cf40a5" (UID: "785beada-f111-44ab-a89c-e3e684cf40a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 05:55:50.003728 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.994147    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-kube-api-access-bw26v" pod "785beada-f111-44ab-a89c-e3e684cf40a5" (UID: "785beada-f111-44ab-a89c-e3e684cf40a5"). InnerVolumeSpecName "kube-api-access-bw26v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:55:50.003728 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:49.996773    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-istio-token" pod "785beada-f111-44ab-a89c-e3e684cf40a5" (UID: "785beada-f111-44ab-a89c-e3e684cf40a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:55:50.003728 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.000797    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-envoy" pod "785beada-f111-44ab-a89c-e3e684cf40a5" (UID: "785beada-f111-44ab-a89c-e3e684cf40a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:55:50.035611 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.031940    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"]
Oct 30 05:55:50.035611 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.032085    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.091905 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.088614    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/785beada-f111-44ab-a89c-e3e684cf40a5-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:50.091905 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.088667    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/785beada-f111-44ab-a89c-e3e684cf40a5-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:50.091905 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.088685    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw26v\" (UniqueName: \"kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-kube-api-access-bw26v\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:50.091905 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.088702    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/785beada-f111-44ab-a89c-e3e684cf40a5-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:55:50.189545 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.189505    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.189736 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.189678    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.189879 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.189845    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.189960 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.189943    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.190025 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.190011    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.190079 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.190047    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6792a38e-911d-4406-8c1a-30bf2bbfddb8-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.190129 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.190088    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-989jx\" (UniqueName: \"kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-kube-api-access-989jx\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.277386 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.277351    5066 generic.go:356] "Generic (PLEG): container finished" podID="785beada-f111-44ab-a89c-e3e684cf40a5" containerID="33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7" exitCode=0
Oct 30 05:55:50.277655 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.277434    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"
Oct 30 05:55:50.277655 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.277438    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz" event={"ID":"785beada-f111-44ab-a89c-e3e684cf40a5","Type":"ContainerDied","Data":"33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7"}
Oct 30 05:55:50.277655 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.277498    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz" event={"ID":"785beada-f111-44ab-a89c-e3e684cf40a5","Type":"ContainerDied","Data":"1e72f44bbb0081dfbf732ae5cfbcc9db7e0b440ddc80d633460ed8092f205544"}
Oct 30 05:55:50.277655 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.277522    5066 scope.go:122] "RemoveContainer" containerID="33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7"
Oct 30 05:55:50.288920 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.288901    5066 scope.go:122] "RemoveContainer" containerID="33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7"
Oct 30 05:55:50.289202 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:55:50.289184    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7\": container with ID starting with 33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7 not found: ID does not exist" containerID="33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7"
Oct 30 05:55:50.289255 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.289210    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7"} err="failed to get container status \"33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7\": rpc error: code = NotFound desc = could not find container \"33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7\": container with ID starting with 33165f6011bc0111b8f02ce861b01d653f834ab9fab59abde505d09eaf47f9e7 not found: ID does not exist"
Oct 30 05:55:50.291001 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.290982    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.291070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.291035    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.291126 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.291086    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.291126 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.291114    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6792a38e-911d-4406-8c1a-30bf2bbfddb8-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.291238 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.291161    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-989jx\" (UniqueName: \"kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-kube-api-access-989jx\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.291289 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.291252    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.291386 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.291360    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.292018 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.291996    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.293567 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.293546    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.293567 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.293640    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.294080 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.294059    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6792a38e-911d-4406-8c1a-30bf2bbfddb8-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.294151 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.294090    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.316051 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.316026    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.316284 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.316267    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-989jx\" (UniqueName: \"kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-kube-api-access-989jx\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6rrhv\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.337176 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.337151    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"]
Oct 30 05:55:50.341070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.341042    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1b2243ef-openshift-default-88dfb6485-d8vdz"]
Oct 30 05:55:50.358887 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.356346    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:50.371185 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.371128    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:50.406103 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.406069    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="785beada-f111-44ab-a89c-e3e684cf40a5" path="/var/lib/kubelet/pods/785beada-f111-44ab-a89c-e3e684cf40a5/volumes"
Oct 30 05:55:50.727484 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.727442    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"]
Oct 30 05:55:50.727646 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:55:50.727551    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6792a38e_911d_4406_8c1a_30bf2bbfddb8.slice/crio-46d6059064dd39f9e4b9423db8acc54daa45819db2db0a199ab7e3701785cc29 WatchSource:0}: Error finding container 46d6059064dd39f9e4b9423db8acc54daa45819db2db0a199ab7e3701785cc29: Status 404 returned error can't find the container with id 46d6059064dd39f9e4b9423db8acc54daa45819db2db0a199ab7e3701785cc29
Oct 30 05:55:50.730500 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.730466    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:55:50.730659 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:50.730541    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 05:55:51.293658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:51.293625    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv" event={"ID":"6792a38e-911d-4406-8c1a-30bf2bbfddb8","Type":"ContainerStarted","Data":"6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b"}
Oct 30 05:55:51.294217 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:51.293683    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv" event={"ID":"6792a38e-911d-4406-8c1a-30bf2bbfddb8","Type":"ContainerStarted","Data":"46d6059064dd39f9e4b9423db8acc54daa45819db2db0a199ab7e3701785cc29"}
Oct 30 05:55:51.294217 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:51.293902    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:51.296215 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:51.296189    5066 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-6rrhv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 05:55:51.296343 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:51.296237    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv" podUID="6792a38e-911d-4406-8c1a-30bf2bbfddb8" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 05:55:51.346604 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:51.346527    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv" podStartSLOduration=7.346508822 podStartE2EDuration="7.346508822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:55:51.344176901 +0000 UTC m=+1455.089462233" watchObservedRunningTime="2026-10-30 05:55:51.346508822 +0000 UTC m=+1455.091794153"
Oct 30 05:55:52.236069 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:52.236018    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:52.307018 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:52.306986    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 05:55:52.407917 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:52.407146    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:52.600947 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:52.600856    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:54.176950 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:54.176917    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:55.852835 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:55.852801    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:57.357951 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:57.357919    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:55:58.987255 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:55:58.987184    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:00.524067 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:00.524023    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:02.076660 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:02.076610    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:02.268740 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:02.268699    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:02.554219 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:56:02.554184    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:56:03.184827 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:03.184791    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:03.348115 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:03.348076    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:03.900680 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:03.900652    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:05.458157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:05.458116    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:06.972071 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:06.972031    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:08.467116 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:08.467037    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:10.126869 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:10.126807    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:11.771768 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:11.771738    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:13.264450 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:13.264415    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:14.841888 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:14.841847    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:16.359900 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:16.359843    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:18.318795 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:18.318758    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:20.088188 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:20.088159    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:21.829687 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:21.829647    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:23.858387 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:23.858338    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:25.358868 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:25.358825    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:26.860957 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:26.860904    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:28.500953 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:28.500903    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:29.047412 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:29.047383    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:29.222100 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:29.222061    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:30.060057 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:30.060010    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:31.535062 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:31.535000    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:32.553464 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:56:32.553427    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:56:33.306026 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:33.305986    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:34.813437 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:34.813406    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:36.388677 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:36.388643    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:36.630846 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:36.630819    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:56:36.636336 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:36.636310    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 05:56:38.151650 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:38.151552    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:39.659315 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:39.659269    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:41.416170 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:41.416127    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:43.133127 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:43.133069    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:44.706934 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:44.706888    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:46.248919 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:46.248880    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:47.947830 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:47.947787    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:49.432648 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:49.432545    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:51.067252 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:51.067217    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:52.734656 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:52.734623    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:54.245301 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:54.243561    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:55.823792 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:55.823752    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:57.560959 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:57.560923    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:56:59.327802 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:56:59.327760    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:01.083675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:01.083633    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:01.534997 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:01.534964    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:02.426410 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:02.426371    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:02.602701 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:02.600979    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:02.689667 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:57:02.689155    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:57:03.104337 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:03.104264    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:03.340976 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:03.340927    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:05.189492 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:05.189451    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:07.020289 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:07.020256    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:09.007084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:09.007046    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:10.583741 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:10.583703    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:11.090679 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:11.090648    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:12.561924 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:12.561882    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"]
Oct 30 05:57:12.562505 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:12.562081    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx" podUID="95858575-99c8-4f01-b9c4-46d9d3e325b3" containerName="maas-api" containerID="cri-o://57ec19c9b0e6cb0b7bf5e05def2387ec9606b11474808a89d5b98eef639e838c" gracePeriod=30
Oct 30 05:57:12.676814 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:12.676782    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:13.054640 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.054551    5066 generic.go:356] "Generic (PLEG): container finished" podID="95858575-99c8-4f01-b9c4-46d9d3e325b3" containerID="57ec19c9b0e6cb0b7bf5e05def2387ec9606b11474808a89d5b98eef639e838c" exitCode=0
Oct 30 05:57:13.054852 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.054691    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx" event={"ID":"95858575-99c8-4f01-b9c4-46d9d3e325b3","Type":"ContainerDied","Data":"57ec19c9b0e6cb0b7bf5e05def2387ec9606b11474808a89d5b98eef639e838c"}
Oct 30 05:57:13.054999 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.054983    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx" event={"ID":"95858575-99c8-4f01-b9c4-46d9d3e325b3","Type":"ContainerDied","Data":"262ce46c4bee4e6a35d97f325df47d17ae0d9190e8981b5d5b3f933dd5c768d0"}
Oct 30 05:57:13.055113 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.055100    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="262ce46c4bee4e6a35d97f325df47d17ae0d9190e8981b5d5b3f933dd5c768d0"
Oct 30 05:57:13.061851 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.061826    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:57:13.163994 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.163957    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-maas-api-tls\") pod \"95858575-99c8-4f01-b9c4-46d9d3e325b3\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") "
Oct 30 05:57:13.171745 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.171710    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-metrics-certs\") pod \"95858575-99c8-4f01-b9c4-46d9d3e325b3\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") "
Oct 30 05:57:13.173279 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.172122    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"]
Oct 30 05:57:13.187139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.173333    5066 cpu_manager.go:414] "containerMap: removing container" podUID="95858575-99c8-4f01-b9c4-46d9d3e325b3" containerName="maas-api"
Oct 30 05:57:13.187139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.173355    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95858575-99c8-4f01-b9c4-46d9d3e325b3" containerName="maas-api"
Oct 30 05:57:13.187139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.173615    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="95858575-99c8-4f01-b9c4-46d9d3e325b3" containerName="maas-api"
Oct 30 05:57:13.187139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.181417    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"]
Oct 30 05:57:13.187139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.181543    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.187139 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.186750    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-maas-api-tls" pod "95858575-99c8-4f01-b9c4-46d9d3e325b3" (UID: "95858575-99c8-4f01-b9c4-46d9d3e325b3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:57:13.191826 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.187743    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95858575-99c8-4f01-b9c4-46d9d3e325b3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95858575-99c8-4f01-b9c4-46d9d3e325b3-openshift-service-ca\") pod \"95858575-99c8-4f01-b9c4-46d9d3e325b3\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") "
Oct 30 05:57:13.191826 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.188093    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95858575-99c8-4f01-b9c4-46d9d3e325b3-openshift-service-ca" pod "95858575-99c8-4f01-b9c4-46d9d3e325b3" (UID: "95858575-99c8-4f01-b9c4-46d9d3e325b3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:57:13.191826 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.188212    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95858575-99c8-4f01-b9c4-46d9d3e325b3-kube-api-access-fgvmz\" (UniqueName: \"kubernetes.io/projected/95858575-99c8-4f01-b9c4-46d9d3e325b3-kube-api-access-fgvmz\") pod \"95858575-99c8-4f01-b9c4-46d9d3e325b3\" (UID: \"95858575-99c8-4f01-b9c4-46d9d3e325b3\") "
Oct 30 05:57:13.191826 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.189566    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:13.191826 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.189609    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95858575-99c8-4f01-b9c4-46d9d3e325b3-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:13.194675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.194562    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-metrics-certs" pod "95858575-99c8-4f01-b9c4-46d9d3e325b3" (UID: "95858575-99c8-4f01-b9c4-46d9d3e325b3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:57:13.198238 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.197250    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95858575-99c8-4f01-b9c4-46d9d3e325b3-kube-api-access-fgvmz" pod "95858575-99c8-4f01-b9c4-46d9d3e325b3" (UID: "95858575-99c8-4f01-b9c4-46d9d3e325b3"). InnerVolumeSpecName "kube-api-access-fgvmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:57:13.201486 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.201462    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w6-f22a52-serving-cert\""
Oct 30 05:57:13.291623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.291142    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.291623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.291250    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxkwb\" (UniqueName: \"kubernetes.io/projected/aaf0009d-4301-46f7-9427-47e17c7a47d6-kube-api-access-jxkwb\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.291623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.291354    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaf0009d-4301-46f7-9427-47e17c7a47d6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.291623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.291372    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.291623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.291520    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgvmz\" (UniqueName: \"kubernetes.io/projected/95858575-99c8-4f01-b9c4-46d9d3e325b3-kube-api-access-fgvmz\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:13.291623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.291532    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95858575-99c8-4f01-b9c4-46d9d3e325b3-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:13.393747 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.393692    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaf0009d-4301-46f7-9427-47e17c7a47d6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.394020 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.394001    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.394606 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.394549    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.394868 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.394850    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxkwb\" (UniqueName: \"kubernetes.io/projected/aaf0009d-4301-46f7-9427-47e17c7a47d6-kube-api-access-jxkwb\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.425076 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.423405    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaf0009d-4301-46f7-9427-47e17c7a47d6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.436693 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.429088    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.462714 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.449657    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.462714 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.449717    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxkwb\" (UniqueName: \"kubernetes.io/projected/aaf0009d-4301-46f7-9427-47e17c7a47d6-kube-api-access-jxkwb\") pod \"maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.481868 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.481776    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:13.569130 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.569089    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:13.710388 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.710359    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:13.937857 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.937829    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:13.962237 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.962209    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"]
Oct 30 05:57:13.962801 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:57:13.962777    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf0009d_4301_46f7_9427_47e17c7a47d6.slice/crio-86d0a6c630125fc2f0da418f563d593f8f16e733d18e2ba65043696e20f54d8e WatchSource:0}: Error finding container 86d0a6c630125fc2f0da418f563d593f8f16e733d18e2ba65043696e20f54d8e: Status 404 returned error can't find the container with id 86d0a6c630125fc2f0da418f563d593f8f16e733d18e2ba65043696e20f54d8e
Oct 30 05:57:13.964726 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:13.964701    5066 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 05:57:14.063791 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:14.063755    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt" event={"ID":"aaf0009d-4301-46f7-9427-47e17c7a47d6","Type":"ContainerStarted","Data":"86d0a6c630125fc2f0da418f563d593f8f16e733d18e2ba65043696e20f54d8e"}
Oct 30 05:57:14.063791 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:14.063792    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"
Oct 30 05:57:14.157970 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:14.157939    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"]
Oct 30 05:57:14.172479 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:14.172449    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1b2243ef-54bff4576-k46cx"]
Oct 30 05:57:14.405218 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:14.405194    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95858575-99c8-4f01-b9c4-46d9d3e325b3" path="/var/lib/kubelet/pods/95858575-99c8-4f01-b9c4-46d9d3e325b3/volumes"
Oct 30 05:57:15.073987 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:15.073954    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt" event={"ID":"aaf0009d-4301-46f7-9427-47e17c7a47d6","Type":"ContainerStarted","Data":"9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411"}
Oct 30 05:57:15.074388 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:15.074019    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:15.122907 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:15.122841    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt" podStartSLOduration=90.691922797 podStartE2EDuration="1m31.122822231s" totalImagesPullingTime="430.899434ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:55:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:57:15.121201175 +0000 UTC m=+1538.866486530" watchObservedRunningTime="2026-10-30 05:57:15.122822231 +0000 UTC m=+1538.868107565"
Oct 30 05:57:15.372430 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:15.372325    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:15.980126 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:15.980100    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:16.181853 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:16.181812    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:16.489669 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:16.489571    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:17.231250 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:17.231221    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:17.991784 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:17.991750    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:18.216564 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:18.216488    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:19.695904 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:19.695870    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:21.058947 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:21.058897    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:21.090182 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:21.090155    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 05:57:21.239469 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:21.239437    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:21.573775 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:21.573746    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:21.757204 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:21.757172    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:22.380654 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:22.380616    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:22.663673 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:22.654206    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:23.537021 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:23.536987    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:24.378825 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:24.378775    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:24.542828 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:24.542772    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:25.412138 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:25.411970    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:26.064997 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.064967    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"]
Oct 30 05:57:26.069406 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.069386    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:26.089184 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.089158    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"]
Oct 30 05:57:26.097290 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.097239    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 05:57:26.097453 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.097322    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 05:57:26.197175 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.197140    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88sg8\" (UniqueName: \"kubernetes.io/projected/71e044f4-e77a-49d3-a378-ad7be8df4e10-kube-api-access-88sg8\") pod \"maas-api-revoke-keys-e2e-api-a-3571f8-q526k\" (UID: \"71e044f4-e77a-49d3-a378-ad7be8df4e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:26.197346 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.197259    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3571f8-q526k\" (UID: \"71e044f4-e77a-49d3-a378-ad7be8df4e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:26.197390 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.197348    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3571f8-q526k\" (UID: \"71e044f4-e77a-49d3-a378-ad7be8df4e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:26.299007 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.298969    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88sg8\" (UniqueName: \"kubernetes.io/projected/71e044f4-e77a-49d3-a378-ad7be8df4e10-kube-api-access-88sg8\") pod \"maas-api-revoke-keys-e2e-api-a-3571f8-q526k\" (UID: \"71e044f4-e77a-49d3-a378-ad7be8df4e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:26.299197 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.299061    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3571f8-q526k\" (UID: \"71e044f4-e77a-49d3-a378-ad7be8df4e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:26.299197 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.299140    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3571f8-q526k\" (UID: \"71e044f4-e77a-49d3-a378-ad7be8df4e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:26.299766 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.299742    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3571f8-q526k\" (UID: \"71e044f4-e77a-49d3-a378-ad7be8df4e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:26.301665 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.301648    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3571f8-q526k\" (UID: \"71e044f4-e77a-49d3-a378-ad7be8df4e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:26.324412 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.324356    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88sg8\" (UniqueName: \"kubernetes.io/projected/71e044f4-e77a-49d3-a378-ad7be8df4e10-kube-api-access-88sg8\") pod \"maas-api-revoke-keys-e2e-api-a-3571f8-q526k\" (UID: \"71e044f4-e77a-49d3-a378-ad7be8df4e10\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:26.386146 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.386114    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:26.698639 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:26.698564    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"]
Oct 30 05:57:26.698770 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:57:26.698641    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e044f4_e77a_49d3_a378_ad7be8df4e10.slice/crio-2000e6c9be870c6f8ef450a56ad96bfab19f2308c235f22b92df5d0208890d56 WatchSource:0}: Error finding container 2000e6c9be870c6f8ef450a56ad96bfab19f2308c235f22b92df5d0208890d56: Status 404 returned error can't find the container with id 2000e6c9be870c6f8ef450a56ad96bfab19f2308c235f22b92df5d0208890d56
Oct 30 05:57:27.172206 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:27.172169    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k" event={"ID":"71e044f4-e77a-49d3-a378-ad7be8df4e10","Type":"ContainerStarted","Data":"2000e6c9be870c6f8ef450a56ad96bfab19f2308c235f22b92df5d0208890d56"}
Oct 30 05:57:27.246623 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:27.246572    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:29.047684 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:29.047102    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:29.211375 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:29.211346    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:29.369998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:29.369964    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:30.198460 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:30.198427    5066 generic.go:356] "Generic (PLEG): container finished" podID="71e044f4-e77a-49d3-a378-ad7be8df4e10" containerID="2c7965974a780a441527baf595ae8dae8d0f6338ddf644e7c8a36e7778dc5a4c" exitCode=0
Oct 30 05:57:30.199072 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:30.198497    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k" event={"ID":"71e044f4-e77a-49d3-a378-ad7be8df4e10","Type":"ContainerDied","Data":"2c7965974a780a441527baf595ae8dae8d0f6338ddf644e7c8a36e7778dc5a4c"}
Oct 30 05:57:30.610784 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:30.610743    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:31.383186 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:31.383161    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:31.492640 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:31.492548    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-xks-ca\") pod \"71e044f4-e77a-49d3-a378-ad7be8df4e10\" (UID: \"71e044f4-e77a-49d3-a378-ad7be8df4e10\") "
Oct 30 05:57:31.492850 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:31.492832    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71e044f4-e77a-49d3-a378-ad7be8df4e10-kube-api-access-88sg8\" (UniqueName: \"kubernetes.io/projected/71e044f4-e77a-49d3-a378-ad7be8df4e10-kube-api-access-88sg8\") pod \"71e044f4-e77a-49d3-a378-ad7be8df4e10\" (UID: \"71e044f4-e77a-49d3-a378-ad7be8df4e10\") "
Oct 30 05:57:31.492954 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:31.492929    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-openshift-ca\") pod \"71e044f4-e77a-49d3-a378-ad7be8df4e10\" (UID: \"71e044f4-e77a-49d3-a378-ad7be8df4e10\") "
Oct 30 05:57:31.493924 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:31.493409    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-openshift-ca" pod "71e044f4-e77a-49d3-a378-ad7be8df4e10" (UID: "71e044f4-e77a-49d3-a378-ad7be8df4e10"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:57:31.494213 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:31.494063    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:31.495862 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:31.495831    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e044f4-e77a-49d3-a378-ad7be8df4e10-kube-api-access-88sg8" pod "71e044f4-e77a-49d3-a378-ad7be8df4e10" (UID: "71e044f4-e77a-49d3-a378-ad7be8df4e10"). InnerVolumeSpecName "kube-api-access-88sg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:57:31.497144 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:31.497124    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-xks-ca" pod "71e044f4-e77a-49d3-a378-ad7be8df4e10" (UID: "71e044f4-e77a-49d3-a378-ad7be8df4e10"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:57:31.595811 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:31.595775    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/71e044f4-e77a-49d3-a378-ad7be8df4e10-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:31.595811 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:31.595808    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88sg8\" (UniqueName: \"kubernetes.io/projected/71e044f4-e77a-49d3-a378-ad7be8df4e10-kube-api-access-88sg8\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:32.214517 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:32.214491    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"
Oct 30 05:57:32.214725 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:32.214527    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k" event={"ID":"71e044f4-e77a-49d3-a378-ad7be8df4e10","Type":"ContainerDied","Data":"2000e6c9be870c6f8ef450a56ad96bfab19f2308c235f22b92df5d0208890d56"}
Oct 30 05:57:32.214725 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:32.214581    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2000e6c9be870c6f8ef450a56ad96bfab19f2308c235f22b92df5d0208890d56"
Oct 30 05:57:32.348382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:32.348323    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:32.553074 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:57:32.552996    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:57:34.262958 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:34.262912    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:36.028464 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:36.028427    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:36.559667 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:36.559557    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:37.479786 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:37.479748    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:37.670984 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:37.670949    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:37.999181 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:37.999148    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:38.183113 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:38.183075    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:39.309142 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:39.309109    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:39.493218 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:39.490063    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:41.031656 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.031625    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"]
Oct 30 05:57:41.033863 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.032316    5066 cpu_manager.go:414] "containerMap: removing container" podUID="71e044f4-e77a-49d3-a378-ad7be8df4e10" containerName="revoke-keys"
Oct 30 05:57:41.033863 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.032327    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71e044f4-e77a-49d3-a378-ad7be8df4e10" containerName="revoke-keys"
Oct 30 05:57:41.033863 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.032448    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="71e044f4-e77a-49d3-a378-ad7be8df4e10" containerName="revoke-keys"
Oct 30 05:57:41.036063 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.036047    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:41.055357 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.055329    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"]
Oct 30 05:57:41.064146 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.064124    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 05:57:41.064295 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.064199    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 05:57:41.184530 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.184499    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c06f23b7-sgwb5\" (UID: \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:41.184764 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.184577    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bkdn\" (UniqueName: \"kubernetes.io/projected/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-kube-api-access-2bkdn\") pod \"maas-api-revoke-sub-c06f23b7-sgwb5\" (UID: \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:41.184828 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.184786    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c06f23b7-sgwb5\" (UID: \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:41.286059 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.285969    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c06f23b7-sgwb5\" (UID: \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:41.286059 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.286018    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bkdn\" (UniqueName: \"kubernetes.io/projected/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-kube-api-access-2bkdn\") pod \"maas-api-revoke-sub-c06f23b7-sgwb5\" (UID: \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:41.286304 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.286110    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c06f23b7-sgwb5\" (UID: \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:41.286796 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.286772    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c06f23b7-sgwb5\" (UID: \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:41.289432 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.289398    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c06f23b7-sgwb5\" (UID: \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:41.300978 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.300948    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:41.311953 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.311914    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bkdn\" (UniqueName: \"kubernetes.io/projected/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-kube-api-access-2bkdn\") pod \"maas-api-revoke-sub-c06f23b7-sgwb5\" (UID: \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:41.350633 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.350603    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:41.679502 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.679459    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"]
Oct 30 05:57:41.760003 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:41.759236    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:42.304912 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:42.304839    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5" event={"ID":"1ca003b8-b0bf-4f21-96c4-7651d99e9db6","Type":"ContainerStarted","Data":"a312746d9ec9d417a3224806f9f4302c5b59cbc5756b726a38e8046f9fae5b65"}
Oct 30 05:57:42.304912 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:42.304892    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5" event={"ID":"1ca003b8-b0bf-4f21-96c4-7651d99e9db6","Type":"ContainerStarted","Data":"629f9a15e78f01cf8dedf17bdf72cc866449a588e9f363b5c663cb28da2d016c"}
Oct 30 05:57:42.361909 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:42.361850    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5" podStartSLOduration=1.361835754 podStartE2EDuration="1.361835754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:57:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:57:42.36071941 +0000 UTC m=+1566.106004741" watchObservedRunningTime="2026-10-30 05:57:42.361835754 +0000 UTC m=+1566.107121142"
Oct 30 05:57:42.791287 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:42.791248    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:42.985696 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:42.981888    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:43.187835 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:43.186604    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:43.316201 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:43.316101    5066 generic.go:356] "Generic (PLEG): container finished" podID="1ca003b8-b0bf-4f21-96c4-7651d99e9db6" containerID="a312746d9ec9d417a3224806f9f4302c5b59cbc5756b726a38e8046f9fae5b65" exitCode=0
Oct 30 05:57:43.316821 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:43.316189    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5" event={"ID":"1ca003b8-b0bf-4f21-96c4-7651d99e9db6","Type":"ContainerDied","Data":"a312746d9ec9d417a3224806f9f4302c5b59cbc5756b726a38e8046f9fae5b65"}
Oct 30 05:57:44.498537 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:44.498513    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:44.642236 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:44.642157    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-kube-api-access-2bkdn\" (UniqueName: \"kubernetes.io/projected/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-kube-api-access-2bkdn\") pod \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\" (UID: \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\") "
Oct 30 05:57:44.642389 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:44.642258    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-openshift-ca\") pod \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\" (UID: \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\") "
Oct 30 05:57:44.642389 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:44.642367    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-xks-ca\") pod \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\" (UID: \"1ca003b8-b0bf-4f21-96c4-7651d99e9db6\") "
Oct 30 05:57:44.642736 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:44.642709    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-openshift-ca" pod "1ca003b8-b0bf-4f21-96c4-7651d99e9db6" (UID: "1ca003b8-b0bf-4f21-96c4-7651d99e9db6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:57:44.643073 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:44.643017    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:44.644618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:44.644579    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-kube-api-access-2bkdn" pod "1ca003b8-b0bf-4f21-96c4-7651d99e9db6" (UID: "1ca003b8-b0bf-4f21-96c4-7651d99e9db6"). InnerVolumeSpecName "kube-api-access-2bkdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:57:44.644618 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:44.644604    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-xks-ca" pod "1ca003b8-b0bf-4f21-96c4-7651d99e9db6" (UID: "1ca003b8-b0bf-4f21-96c4-7651d99e9db6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:57:44.744660 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:44.744583    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bkdn\" (UniqueName: \"kubernetes.io/projected/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-kube-api-access-2bkdn\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:44.744660 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:44.744658    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1ca003b8-b0bf-4f21-96c4-7651d99e9db6-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:45.070640 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:45.070567    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:45.331318 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:45.331227    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5" event={"ID":"1ca003b8-b0bf-4f21-96c4-7651d99e9db6","Type":"ContainerDied","Data":"629f9a15e78f01cf8dedf17bdf72cc866449a588e9f363b5c663cb28da2d016c"}
Oct 30 05:57:45.331318 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:45.331279    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="629f9a15e78f01cf8dedf17bdf72cc866449a588e9f363b5c663cb28da2d016c"
Oct 30 05:57:45.331536 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:45.331318    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"
Oct 30 05:57:45.420668 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:45.420627    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:46.517452 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:46.517373    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:46.808189 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:46.808099    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:47.286772 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:47.286698    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:48.974727 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:48.974694    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:50.701162 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:50.701128    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:50.878638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:50.878565    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:51.490748 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:51.490703    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:52.331946 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:52.331901    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:52.979156 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:52.979125    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"]
Oct 30 05:57:52.979341 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:52.979310    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy" containerID="cri-o://1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b" gracePeriod=30
Oct 30 05:57:52.994305 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:52.994276    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"]
Oct 30 05:57:52.994504 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:52.994482    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" podUID="2964d72a-feee-4fcb-8b03-5048919222cb" containerName="istio-proxy" containerID="cri-o://6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c" gracePeriod=30
Oct 30 05:57:53.915250 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:53.915201    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:55.540963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:55.540914    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:57.052842 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:57.052808    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:57.400602 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:57.400508    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:57.679425 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:57.679396    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:58.312545 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.312234    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:57:58.352891 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.352863    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:57:58.374078 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.374051    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-token\" (UniqueName: \"kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-token\") pod \"452c383a-8951-4d4e-abe9-2ba633bed28f\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") "
Oct 30 05:57:58.374239 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.374169    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-credential-socket\") pod \"452c383a-8951-4d4e-abe9-2ba633bed28f\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") "
Oct 30 05:57:58.374299 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.374255    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-socket\") pod \"452c383a-8951-4d4e-abe9-2ba633bed28f\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") "
Oct 30 05:57:58.374299 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.374281    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/452c383a-8951-4d4e-abe9-2ba633bed28f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452c383a-8951-4d4e-abe9-2ba633bed28f-istiod-ca-cert\") pod \"452c383a-8951-4d4e-abe9-2ba633bed28f\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") "
Oct 30 05:57:58.374401 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.374313    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-certs\") pod \"452c383a-8951-4d4e-abe9-2ba633bed28f\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") "
Oct 30 05:57:58.374532 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.374496    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-podinfo\") pod \"452c383a-8951-4d4e-abe9-2ba633bed28f\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") "
Oct 30 05:57:58.374645 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.374537    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-kube-api-access-lrlzk\" (UniqueName: \"kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-kube-api-access-lrlzk\") pod \"452c383a-8951-4d4e-abe9-2ba633bed28f\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") "
Oct 30 05:57:58.374730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.374710    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-data\") pod \"452c383a-8951-4d4e-abe9-2ba633bed28f\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") "
Oct 30 05:57:58.374867 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.374747    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-credential-socket" pod "452c383a-8951-4d4e-abe9-2ba633bed28f" (UID: "452c383a-8951-4d4e-abe9-2ba633bed28f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:57:58.374867 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.374757    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-socket" pod "452c383a-8951-4d4e-abe9-2ba633bed28f" (UID: "452c383a-8951-4d4e-abe9-2ba633bed28f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:57:58.374867 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.374772    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-envoy\") pod \"452c383a-8951-4d4e-abe9-2ba633bed28f\" (UID: \"452c383a-8951-4d4e-abe9-2ba633bed28f\") "
Oct 30 05:57:58.375014 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.374983    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-data" pod "452c383a-8951-4d4e-abe9-2ba633bed28f" (UID: "452c383a-8951-4d4e-abe9-2ba633bed28f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:57:58.375014 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.375000    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/452c383a-8951-4d4e-abe9-2ba633bed28f-istiod-ca-cert" pod "452c383a-8951-4d4e-abe9-2ba633bed28f" (UID: "452c383a-8951-4d4e-abe9-2ba633bed28f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:57:58.375238 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.375208    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-certs" pod "452c383a-8951-4d4e-abe9-2ba633bed28f" (UID: "452c383a-8951-4d4e-abe9-2ba633bed28f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:57:58.375684 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.375655    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.375684 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.375679    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/452c383a-8951-4d4e-abe9-2ba633bed28f-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.375815 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.375694    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.375815 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.375705    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.375815 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.375717    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.377145 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.377124    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-token" pod "452c383a-8951-4d4e-abe9-2ba633bed28f" (UID: "452c383a-8951-4d4e-abe9-2ba633bed28f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:57:58.377313 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.377287    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-podinfo" pod "452c383a-8951-4d4e-abe9-2ba633bed28f" (UID: "452c383a-8951-4d4e-abe9-2ba633bed28f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 05:57:58.377973 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.377951    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-envoy" pod "452c383a-8951-4d4e-abe9-2ba633bed28f" (UID: "452c383a-8951-4d4e-abe9-2ba633bed28f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:57:58.377973 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.377956    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-kube-api-access-lrlzk" pod "452c383a-8951-4d4e-abe9-2ba633bed28f" (UID: "452c383a-8951-4d4e-abe9-2ba633bed28f"). InnerVolumeSpecName "kube-api-access-lrlzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:57:58.463360 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.463324    5066 generic.go:356] "Generic (PLEG): container finished" podID="2964d72a-feee-4fcb-8b03-5048919222cb" containerID="6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c" exitCode=0
Oct 30 05:57:58.463535 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.463523    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" event={"ID":"2964d72a-feee-4fcb-8b03-5048919222cb","Type":"ContainerDied","Data":"6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c"}
Oct 30 05:57:58.463711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.463572    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs" event={"ID":"2964d72a-feee-4fcb-8b03-5048919222cb","Type":"ContainerDied","Data":"9b2dab4ec59b025225ce87d6e5ca17da0aaa54346549727b94933082cd300c4c"}
Oct 30 05:57:58.463711 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.463656    5066 scope.go:122] "RemoveContainer" containerID="6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c"
Oct 30 05:57:58.463843 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.463828    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"
Oct 30 05:57:58.467696 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.467169    5066 generic.go:356] "Generic (PLEG): container finished" podID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerID="1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b" exitCode=0
Oct 30 05:57:58.467696 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.467261    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" event={"ID":"452c383a-8951-4d4e-abe9-2ba633bed28f","Type":"ContainerDied","Data":"1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b"}
Oct 30 05:57:58.467696 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.467301    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm" event={"ID":"452c383a-8951-4d4e-abe9-2ba633bed28f","Type":"ContainerDied","Data":"1fc9429b6f9252f3bb97fece72a66dd97daf9795f854ba768672cf14426e7fec"}
Oct 30 05:57:58.467696 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.467378    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"
Oct 30 05:57:58.476356 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.476330    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2964d72a-feee-4fcb-8b03-5048919222cb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2964d72a-feee-4fcb-8b03-5048919222cb-istio-podinfo\") pod \"2964d72a-feee-4fcb-8b03-5048919222cb\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") "
Oct 30 05:57:58.476500 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.476408    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-data\") pod \"2964d72a-feee-4fcb-8b03-5048919222cb\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") "
Oct 30 05:57:58.476500 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.476473    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-credential-socket\") pod \"2964d72a-feee-4fcb-8b03-5048919222cb\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") "
Oct 30 05:57:58.476651 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.476498    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-istio-token\" (UniqueName: \"kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-istio-token\") pod \"2964d72a-feee-4fcb-8b03-5048919222cb\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") "
Oct 30 05:57:58.476651 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.476527    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2964d72a-feee-4fcb-8b03-5048919222cb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2964d72a-feee-4fcb-8b03-5048919222cb-istiod-ca-cert\") pod \"2964d72a-feee-4fcb-8b03-5048919222cb\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") "
Oct 30 05:57:58.476749 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.476706    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-certs\") pod \"2964d72a-feee-4fcb-8b03-5048919222cb\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") "
Oct 30 05:57:58.476749 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.476738    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-socket\") pod \"2964d72a-feee-4fcb-8b03-5048919222cb\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") "
Oct 30 05:57:58.476847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.476793    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-kube-api-access-r4q2n\" (UniqueName: \"kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-kube-api-access-r4q2n\") pod \"2964d72a-feee-4fcb-8b03-5048919222cb\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") "
Oct 30 05:57:58.476939 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.476921    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-envoy\") pod \"2964d72a-feee-4fcb-8b03-5048919222cb\" (UID: \"2964d72a-feee-4fcb-8b03-5048919222cb\") "
Oct 30 05:57:58.477834 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.477804    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-credential-socket" pod "2964d72a-feee-4fcb-8b03-5048919222cb" (UID: "2964d72a-feee-4fcb-8b03-5048919222cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:57:58.477973 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.477955    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-certs" pod "2964d72a-feee-4fcb-8b03-5048919222cb" (UID: "2964d72a-feee-4fcb-8b03-5048919222cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:57:58.478310 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.478280    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2964d72a-feee-4fcb-8b03-5048919222cb-istiod-ca-cert" pod "2964d72a-feee-4fcb-8b03-5048919222cb" (UID: "2964d72a-feee-4fcb-8b03-5048919222cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:57:58.478412 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.478344    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-data" pod "2964d72a-feee-4fcb-8b03-5048919222cb" (UID: "2964d72a-feee-4fcb-8b03-5048919222cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:57:58.478466 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.478420    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-socket" pod "2964d72a-feee-4fcb-8b03-5048919222cb" (UID: "2964d72a-feee-4fcb-8b03-5048919222cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:57:58.480132 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.480107    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.480224 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.480136    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.480224 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.480151    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.480224 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.480164    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrlzk\" (UniqueName: \"kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-kube-api-access-lrlzk\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.480224 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.480177    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.480224 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.480191    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.480224 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.480203    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.480224 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.480218    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2964d72a-feee-4fcb-8b03-5048919222cb-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.480471 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.480230    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/452c383a-8951-4d4e-abe9-2ba633bed28f-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.481132 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.481107    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-kube-api-access-r4q2n" pod "2964d72a-feee-4fcb-8b03-5048919222cb" (UID: "2964d72a-feee-4fcb-8b03-5048919222cb"). InnerVolumeSpecName "kube-api-access-r4q2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:57:58.482743 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.482705    5066 scope.go:122] "RemoveContainer" containerID="6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c"
Oct 30 05:57:58.483127 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:57:58.483100    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c\": container with ID starting with 6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c not found: ID does not exist" containerID="6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c"
Oct 30 05:57:58.483217 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.483138    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c"} err="failed to get container status \"6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c\": rpc error: code = NotFound desc = could not find container \"6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c\": container with ID starting with 6ba6856728a18f215fdeeaa15dae05b5279be3816e17dff8b298ac6dd74c3c1c not found: ID does not exist"
Oct 30 05:57:58.483217 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.483164    5066 scope.go:122] "RemoveContainer" containerID="1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b"
Oct 30 05:57:58.485940 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.485914    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-envoy" pod "2964d72a-feee-4fcb-8b03-5048919222cb" (UID: "2964d72a-feee-4fcb-8b03-5048919222cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:57:58.486157 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.486135    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-istio-token" pod "2964d72a-feee-4fcb-8b03-5048919222cb" (UID: "2964d72a-feee-4fcb-8b03-5048919222cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:57:58.487319 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.487293    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2964d72a-feee-4fcb-8b03-5048919222cb-istio-podinfo" pod "2964d72a-feee-4fcb-8b03-5048919222cb" (UID: "2964d72a-feee-4fcb-8b03-5048919222cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 05:57:58.494863 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.494766    5066 scope.go:122] "RemoveContainer" containerID="1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b"
Oct 30 05:57:58.495105 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:57:58.495073    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b\": container with ID starting with 1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b not found: ID does not exist" containerID="1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b"
Oct 30 05:57:58.495194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.495103    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b"} err="failed to get container status \"1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b\": rpc error: code = NotFound desc = could not find container \"1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b\": container with ID starting with 1bb17c00d69bab4ba7eacf23a706b2d33935c448792b2a9c19d171158a7b6f5b not found: ID does not exist"
Oct 30 05:57:58.533567 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.533488    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"]
Oct 30 05:57:58.542110 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.542085    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-hfblm"]
Oct 30 05:57:58.581654 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.581622    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.581654 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.581650    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4q2n\" (UniqueName: \"kubernetes.io/projected/2964d72a-feee-4fcb-8b03-5048919222cb-kube-api-access-r4q2n\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.581654 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.581660    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2964d72a-feee-4fcb-8b03-5048919222cb-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.581969 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.581668    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2964d72a-feee-4fcb-8b03-5048919222cb-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:57:58.819130 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.819047    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"]
Oct 30 05:57:58.821753 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.821725    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:58.825528 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:58.825506    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-vfwgs"]
Oct 30 05:57:59.003012 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:59.002963    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:57:59.201838 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:57:59.201806    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:00.404652 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:00.404620    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2964d72a-feee-4fcb-8b03-5048919222cb" path="/var/lib/kubelet/pods/2964d72a-feee-4fcb-8b03-5048919222cb/volumes"
Oct 30 05:58:00.405207 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:00.405192    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" path="/var/lib/kubelet/pods/452c383a-8951-4d4e-abe9-2ba633bed28f/volumes"
Oct 30 05:58:00.643376 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:00.643343    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:02.223043 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:02.222956    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:02.554150 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:58:02.554069    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:58:03.834648 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:03.834557    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:05.523219 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:05.523185    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:07.031346 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:07.031309    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:08.825331 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:08.825227    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:10.877822 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:10.876965    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:11.618394 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:11.617840    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-p99cc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:58:11.618394 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:11.617912    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:58:11.665760 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:11.665711    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xjhtt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 05:58:11.666073 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:11.666022    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 05:58:12.723902 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:12.722923    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:14.218945 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:14.218913    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:15.932325 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:15.932282    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:17.434512 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:17.434450    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:18.964439 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:18.964409    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:19.215909 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:19.215834    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:20.723085 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:20.723049    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:22.203168 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:22.203137    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:23.798612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:23.795541    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:25.424861 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:25.424822    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:25.600257 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:25.600213    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:27.311086 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:27.311033    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:28.944552 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:28.944518    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:29.323995 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:29.323916    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:30.624949 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:30.624913    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:32.126670 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:32.126265    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:32.557302 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:58:32.557270    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:58:33.977314 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:33.977267    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:33.993616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:33.993549    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"]
Oct 30 05:58:33.995421 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:33.995390    5066 cpu_manager.go:414] "containerMap: removing container" podUID="1ca003b8-b0bf-4f21-96c4-7651d99e9db6" containerName="revoke-keys"
Oct 30 05:58:33.995815 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:33.995789    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ca003b8-b0bf-4f21-96c4-7651d99e9db6" containerName="revoke-keys"
Oct 30 05:58:33.995963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:33.995875    5066 cpu_manager.go:414] "containerMap: removing container" podUID="2964d72a-feee-4fcb-8b03-5048919222cb" containerName="istio-proxy"
Oct 30 05:58:33.995963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:33.995886    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2964d72a-feee-4fcb-8b03-5048919222cb" containerName="istio-proxy"
Oct 30 05:58:33.995963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:33.995925    5066 cpu_manager.go:414] "containerMap: removing container" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy"
Oct 30 05:58:33.995963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:33.995932    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy"
Oct 30 05:58:33.996171 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:33.996119    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="452c383a-8951-4d4e-abe9-2ba633bed28f" containerName="istio-proxy"
Oct 30 05:58:33.996171 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:33.996141    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ca003b8-b0bf-4f21-96c4-7651d99e9db6" containerName="revoke-keys"
Oct 30 05:58:33.996171 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:33.996152    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="2964d72a-feee-4fcb-8b03-5048919222cb" containerName="istio-proxy"
Oct 30 05:58:34.006643 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.003270    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:34.028613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.028247    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 05:58:34.028613 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.028447    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 05:58:34.049917 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.049885    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"]
Oct 30 05:58:34.058722 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.058689    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"]
Oct 30 05:58:34.064555 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.064523    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:34.084574 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.084539    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"]
Oct 30 05:58:34.099913 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.099874    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z\" (UID: \"5df1dbc1-9eac-4584-ade8-d232e51578b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:34.100174 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.100143    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmsht\" (UniqueName: \"kubernetes.io/projected/5df1dbc1-9eac-4584-ade8-d232e51578b1-kube-api-access-bmsht\") pod \"maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z\" (UID: \"5df1dbc1-9eac-4584-ade8-d232e51578b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:34.100307 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.100220    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z\" (UID: \"5df1dbc1-9eac-4584-ade8-d232e51578b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:34.100307 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.100261    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7\" (UID: \"a87c9be5-b7ef-4639-9551-c4edd7913d04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:34.100423 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.100352    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7\" (UID: \"a87c9be5-b7ef-4639-9551-c4edd7913d04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:34.100514 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.100498    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k99f\" (UniqueName: \"kubernetes.io/projected/a87c9be5-b7ef-4639-9551-c4edd7913d04-kube-api-access-4k99f\") pod \"maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7\" (UID: \"a87c9be5-b7ef-4639-9551-c4edd7913d04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:34.201944 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.201910    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmsht\" (UniqueName: \"kubernetes.io/projected/5df1dbc1-9eac-4584-ade8-d232e51578b1-kube-api-access-bmsht\") pod \"maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z\" (UID: \"5df1dbc1-9eac-4584-ade8-d232e51578b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:34.202151 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.201963    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z\" (UID: \"5df1dbc1-9eac-4584-ade8-d232e51578b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:34.202151 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.201982    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7\" (UID: \"a87c9be5-b7ef-4639-9551-c4edd7913d04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:34.202151 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.202015    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7\" (UID: \"a87c9be5-b7ef-4639-9551-c4edd7913d04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:34.202151 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.202139    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k99f\" (UniqueName: \"kubernetes.io/projected/a87c9be5-b7ef-4639-9551-c4edd7913d04-kube-api-access-4k99f\") pod \"maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7\" (UID: \"a87c9be5-b7ef-4639-9551-c4edd7913d04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:34.202382 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.202202    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z\" (UID: \"5df1dbc1-9eac-4584-ade8-d232e51578b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:34.202736 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.202699    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z\" (UID: \"5df1dbc1-9eac-4584-ade8-d232e51578b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:34.202857 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.202840    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7\" (UID: \"a87c9be5-b7ef-4639-9551-c4edd7913d04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:34.204534 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.204512    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7\" (UID: \"a87c9be5-b7ef-4639-9551-c4edd7913d04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:34.204690 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.204675    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z\" (UID: \"5df1dbc1-9eac-4584-ade8-d232e51578b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:34.228498 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.228436    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmsht\" (UniqueName: \"kubernetes.io/projected/5df1dbc1-9eac-4584-ade8-d232e51578b1-kube-api-access-bmsht\") pod \"maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z\" (UID: \"5df1dbc1-9eac-4584-ade8-d232e51578b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:34.228716 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.228609    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k99f\" (UniqueName: \"kubernetes.io/projected/a87c9be5-b7ef-4639-9551-c4edd7913d04-kube-api-access-4k99f\") pod \"maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7\" (UID: \"a87c9be5-b7ef-4639-9551-c4edd7913d04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:34.347939 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.347907    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:34.383817 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.383782    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:34.663001 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.662967    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"]
Oct 30 05:58:34.663001 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:58:34.662987    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda87c9be5_b7ef_4639_9551_c4edd7913d04.slice/crio-ed36c35f2f3a19aee84130604f7f17cdd3a8c16bad8325fee7ef252b71024014 WatchSource:0}: Error finding container ed36c35f2f3a19aee84130604f7f17cdd3a8c16bad8325fee7ef252b71024014: Status 404 returned error can't find the container with id ed36c35f2f3a19aee84130604f7f17cdd3a8c16bad8325fee7ef252b71024014
Oct 30 05:58:34.714656 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:58:34.714628    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df1dbc1_9eac_4584_ade8_d232e51578b1.slice/crio-9656c2e29a08b392211931e08bd06b0d429c7e37e974b3560de5065bc4cc48a3 WatchSource:0}: Error finding container 9656c2e29a08b392211931e08bd06b0d429c7e37e974b3560de5065bc4cc48a3: Status 404 returned error can't find the container with id 9656c2e29a08b392211931e08bd06b0d429c7e37e974b3560de5065bc4cc48a3
Oct 30 05:58:34.716851 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.716823    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"]
Oct 30 05:58:34.833525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.833495    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z" event={"ID":"5df1dbc1-9eac-4584-ade8-d232e51578b1","Type":"ContainerStarted","Data":"9656c2e29a08b392211931e08bd06b0d429c7e37e974b3560de5065bc4cc48a3"}
Oct 30 05:58:34.835185 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:34.835161    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7" event={"ID":"a87c9be5-b7ef-4639-9551-c4edd7913d04","Type":"ContainerStarted","Data":"ed36c35f2f3a19aee84130604f7f17cdd3a8c16bad8325fee7ef252b71024014"}
Oct 30 05:58:35.847745 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:35.847704    5066 generic.go:356] "Generic (PLEG): container finished" podID="a87c9be5-b7ef-4639-9551-c4edd7913d04" containerID="5040c5cba5ce54111a0e82468ae1fe484368c83e215c907969e65d5bb24f9637" exitCode=0
Oct 30 05:58:35.848225 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:35.847836    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7" event={"ID":"a87c9be5-b7ef-4639-9551-c4edd7913d04","Type":"ContainerDied","Data":"5040c5cba5ce54111a0e82468ae1fe484368c83e215c907969e65d5bb24f9637"}
Oct 30 05:58:35.850462 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:35.850422    5066 generic.go:356] "Generic (PLEG): container finished" podID="5df1dbc1-9eac-4584-ade8-d232e51578b1" containerID="b7918a8b8a753ee2f2e40e36a301acc257168ec2a85d0d35a13f062fc9be9f66" exitCode=0
Oct 30 05:58:35.850652 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:35.850464    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z" event={"ID":"5df1dbc1-9eac-4584-ade8-d232e51578b1","Type":"ContainerDied","Data":"b7918a8b8a753ee2f2e40e36a301acc257168ec2a85d0d35a13f062fc9be9f66"}
Oct 30 05:58:35.901508 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:35.901471    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:37.060292 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.060264    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:37.120690 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.120660    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:37.153815 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.153784    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a87c9be5-b7ef-4639-9551-c4edd7913d04-kube-api-access-4k99f\" (UniqueName: \"kubernetes.io/projected/a87c9be5-b7ef-4639-9551-c4edd7913d04-kube-api-access-4k99f\") pod \"a87c9be5-b7ef-4639-9551-c4edd7913d04\" (UID: \"a87c9be5-b7ef-4639-9551-c4edd7913d04\") "
Oct 30 05:58:37.153998 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.153886    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-openshift-ca\") pod \"a87c9be5-b7ef-4639-9551-c4edd7913d04\" (UID: \"a87c9be5-b7ef-4639-9551-c4edd7913d04\") "
Oct 30 05:58:37.154271 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.154247    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-xks-ca\") pod \"a87c9be5-b7ef-4639-9551-c4edd7913d04\" (UID: \"a87c9be5-b7ef-4639-9551-c4edd7913d04\") "
Oct 30 05:58:37.154450 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.154427    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-openshift-ca" pod "a87c9be5-b7ef-4639-9551-c4edd7913d04" (UID: "a87c9be5-b7ef-4639-9551-c4edd7913d04"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:58:37.155138 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.155105    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:58:37.157059 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.157032    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-xks-ca" pod "a87c9be5-b7ef-4639-9551-c4edd7913d04" (UID: "a87c9be5-b7ef-4639-9551-c4edd7913d04"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:58:37.157156 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.157061    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a87c9be5-b7ef-4639-9551-c4edd7913d04-kube-api-access-4k99f" pod "a87c9be5-b7ef-4639-9551-c4edd7913d04" (UID: "a87c9be5-b7ef-4639-9551-c4edd7913d04"). InnerVolumeSpecName "kube-api-access-4k99f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:58:37.256466 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.256423    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5df1dbc1-9eac-4584-ade8-d232e51578b1-kube-api-access-bmsht\" (UniqueName: \"kubernetes.io/projected/5df1dbc1-9eac-4584-ade8-d232e51578b1-kube-api-access-bmsht\") pod \"5df1dbc1-9eac-4584-ade8-d232e51578b1\" (UID: \"5df1dbc1-9eac-4584-ade8-d232e51578b1\") "
Oct 30 05:58:37.256679 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.256543    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-openshift-ca\") pod \"5df1dbc1-9eac-4584-ade8-d232e51578b1\" (UID: \"5df1dbc1-9eac-4584-ade8-d232e51578b1\") "
Oct 30 05:58:37.256679 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.256654    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-xks-ca\") pod \"5df1dbc1-9eac-4584-ade8-d232e51578b1\" (UID: \"5df1dbc1-9eac-4584-ade8-d232e51578b1\") "
Oct 30 05:58:37.256969 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.256940    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-openshift-ca" pod "5df1dbc1-9eac-4584-ade8-d232e51578b1" (UID: "5df1dbc1-9eac-4584-ade8-d232e51578b1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:58:37.257271 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.257185    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a87c9be5-b7ef-4639-9551-c4edd7913d04-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:58:37.257271 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.257204    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4k99f\" (UniqueName: \"kubernetes.io/projected/a87c9be5-b7ef-4639-9551-c4edd7913d04-kube-api-access-4k99f\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:58:37.257271 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.257214    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:58:37.258867 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.258847    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5df1dbc1-9eac-4584-ade8-d232e51578b1-kube-api-access-bmsht" pod "5df1dbc1-9eac-4584-ade8-d232e51578b1" (UID: "5df1dbc1-9eac-4584-ade8-d232e51578b1"). InnerVolumeSpecName "kube-api-access-bmsht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:58:37.259197 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.259178    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-xks-ca" pod "5df1dbc1-9eac-4584-ade8-d232e51578b1" (UID: "5df1dbc1-9eac-4584-ade8-d232e51578b1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:58:37.358731 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.358644    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmsht\" (UniqueName: \"kubernetes.io/projected/5df1dbc1-9eac-4584-ade8-d232e51578b1-kube-api-access-bmsht\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:58:37.358731 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.358672    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5df1dbc1-9eac-4584-ade8-d232e51578b1-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:58:37.577060 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.577021    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:37.845378 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.845342    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:37.873465 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.873437    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"
Oct 30 05:58:37.873465 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.873450    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7" event={"ID":"a87c9be5-b7ef-4639-9551-c4edd7913d04","Type":"ContainerDied","Data":"ed36c35f2f3a19aee84130604f7f17cdd3a8c16bad8325fee7ef252b71024014"}
Oct 30 05:58:37.873745 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.873525    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed36c35f2f3a19aee84130604f7f17cdd3a8c16bad8325fee7ef252b71024014"
Oct 30 05:58:37.875042 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.875019    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z" event={"ID":"5df1dbc1-9eac-4584-ade8-d232e51578b1","Type":"ContainerDied","Data":"9656c2e29a08b392211931e08bd06b0d429c7e37e974b3560de5065bc4cc48a3"}
Oct 30 05:58:37.875178 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.875048    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"
Oct 30 05:58:37.875178 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:37.875059    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9656c2e29a08b392211931e08bd06b0d429c7e37e974b3560de5065bc4cc48a3"
Oct 30 05:58:39.499952 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:39.499923    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:39.918908 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:39.918870    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:41.486847 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:41.486808    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:42.836025 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:42.835991    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:43.523836 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:43.523805    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:43.696164 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:43.696074    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:43.861582 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:43.861491    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:44.157921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:44.157887    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:44.581839 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:44.581757    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:44.789124 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:44.789083    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:45.367662 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:45.367633    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:45.549149 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:45.549099    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:46.901135 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:46.901092    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:47.453161 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:47.453071    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:47.646710 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:47.646624    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:47.779638 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:47.779518    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"]
Oct 30 05:58:47.793717 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:47.793681    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3571f8-q526k"]
Oct 30 05:58:47.891670 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:47.891639    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:48.270942 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:48.270890    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:48.358084 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:48.358049    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"]
Oct 30 05:58:48.371625 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:48.371569    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a3efab09-xmw9z"]
Oct 30 05:58:48.404221 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:48.404185    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5df1dbc1-9eac-4584-ade8-d232e51578b1" path="/var/lib/kubelet/pods/5df1dbc1-9eac-4584-ade8-d232e51578b1/volumes"
Oct 30 05:58:48.404940 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:48.404922    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71e044f4-e77a-49d3-a378-ad7be8df4e10" path="/var/lib/kubelet/pods/71e044f4-e77a-49d3-a378-ad7be8df4e10/volumes"
Oct 30 05:58:48.444256 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:48.444228    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:48.626628 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:48.626506    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:48.946960 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:48.946921    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:50.319921 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:50.319885    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:52.114061 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:52.113996    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:52.290097 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:52.290064    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:52.503227 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:52.503188    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:53.007362 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.007319    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"]
Oct 30 05:58:53.008075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.008055    5066 cpu_manager.go:414] "containerMap: removing container" podUID="5df1dbc1-9eac-4584-ade8-d232e51578b1" containerName="revoke-keys"
Oct 30 05:58:53.008075 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.008075    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5df1dbc1-9eac-4584-ade8-d232e51578b1" containerName="revoke-keys"
Oct 30 05:58:53.008256 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.008142    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a87c9be5-b7ef-4639-9551-c4edd7913d04" containerName="revoke-keys"
Oct 30 05:58:53.008256 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.008151    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a87c9be5-b7ef-4639-9551-c4edd7913d04" containerName="revoke-keys"
Oct 30 05:58:53.008365 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.008351    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a87c9be5-b7ef-4639-9551-c4edd7913d04" containerName="revoke-keys"
Oct 30 05:58:53.008425 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.008375    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="5df1dbc1-9eac-4584-ade8-d232e51578b1" containerName="revoke-keys"
Oct 30 05:58:53.013717 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.013697    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:53.033510 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.033486    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 05:58:53.033757 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.033738    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 05:58:53.072697 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.072668    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"]
Oct 30 05:58:53.086965 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.086930    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs\" (UID: \"22d42849-16e2-4f88-93b8-847b71eadaf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:53.087196 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.087123    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs\" (UID: \"22d42849-16e2-4f88-93b8-847b71eadaf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:53.087260 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.087219    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l7s2\" (UniqueName: \"kubernetes.io/projected/22d42849-16e2-4f88-93b8-847b71eadaf2-kube-api-access-6l7s2\") pod \"maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs\" (UID: \"22d42849-16e2-4f88-93b8-847b71eadaf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:53.188983 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.188951    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l7s2\" (UniqueName: \"kubernetes.io/projected/22d42849-16e2-4f88-93b8-847b71eadaf2-kube-api-access-6l7s2\") pod \"maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs\" (UID: \"22d42849-16e2-4f88-93b8-847b71eadaf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:53.189425 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.189098    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs\" (UID: \"22d42849-16e2-4f88-93b8-847b71eadaf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:53.189425 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.189236    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs\" (UID: \"22d42849-16e2-4f88-93b8-847b71eadaf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:53.189820 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.189802    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs\" (UID: \"22d42849-16e2-4f88-93b8-847b71eadaf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:53.191771 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.191749    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs\" (UID: \"22d42849-16e2-4f88-93b8-847b71eadaf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:53.217724 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.217695    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l7s2\" (UniqueName: \"kubernetes.io/projected/22d42849-16e2-4f88-93b8-847b71eadaf2-kube-api-access-6l7s2\") pod \"maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs\" (UID: \"22d42849-16e2-4f88-93b8-847b71eadaf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:53.330252 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.330177    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:53.649127 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:58:53.649081    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d42849_16e2_4f88_93b8_847b71eadaf2.slice/crio-4001a23220ae43c082c6777ae4306aab12d3b787573ca544d155c09d4a5382f2 WatchSource:0}: Error finding container 4001a23220ae43c082c6777ae4306aab12d3b787573ca544d155c09d4a5382f2: Status 404 returned error can't find the container with id 4001a23220ae43c082c6777ae4306aab12d3b787573ca544d155c09d4a5382f2
Oct 30 05:58:53.649879 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:53.649836    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"]
Oct 30 05:58:54.056072 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:54.056028    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs" event={"ID":"22d42849-16e2-4f88-93b8-847b71eadaf2","Type":"ContainerStarted","Data":"4001a23220ae43c082c6777ae4306aab12d3b787573ca544d155c09d4a5382f2"}
Oct 30 05:58:54.353675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:54.352441    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:55.064859 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:55.064765    5066 generic.go:356] "Generic (PLEG): container finished" podID="22d42849-16e2-4f88-93b8-847b71eadaf2" containerID="f8874f3d124647b79f1087f848a4ac4b5259f011c46552833d14de3cb2e1eef9" exitCode=0
Oct 30 05:58:55.065019 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:55.064852    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs" event={"ID":"22d42849-16e2-4f88-93b8-847b71eadaf2","Type":"ContainerDied","Data":"f8874f3d124647b79f1087f848a4ac4b5259f011c46552833d14de3cb2e1eef9"}
Oct 30 05:58:56.249615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.242066    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:56.499072 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.499042    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:56.607835 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.607794    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:56.685408 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.685360    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-openshift-ca\") pod \"22d42849-16e2-4f88-93b8-847b71eadaf2\" (UID: \"22d42849-16e2-4f88-93b8-847b71eadaf2\") "
Oct 30 05:58:56.685408 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.685406    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22d42849-16e2-4f88-93b8-847b71eadaf2-kube-api-access-6l7s2\" (UniqueName: \"kubernetes.io/projected/22d42849-16e2-4f88-93b8-847b71eadaf2-kube-api-access-6l7s2\") pod \"22d42849-16e2-4f88-93b8-847b71eadaf2\" (UID: \"22d42849-16e2-4f88-93b8-847b71eadaf2\") "
Oct 30 05:58:56.685743 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.685534    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-xks-ca\") pod \"22d42849-16e2-4f88-93b8-847b71eadaf2\" (UID: \"22d42849-16e2-4f88-93b8-847b71eadaf2\") "
Oct 30 05:58:56.685815 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.685783    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-openshift-ca" pod "22d42849-16e2-4f88-93b8-847b71eadaf2" (UID: "22d42849-16e2-4f88-93b8-847b71eadaf2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:58:56.686476 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.686442    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:58:56.687832 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.687802    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d42849-16e2-4f88-93b8-847b71eadaf2-kube-api-access-6l7s2" pod "22d42849-16e2-4f88-93b8-847b71eadaf2" (UID: "22d42849-16e2-4f88-93b8-847b71eadaf2"). InnerVolumeSpecName "kube-api-access-6l7s2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:58:56.687934 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.687874    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-xks-ca" pod "22d42849-16e2-4f88-93b8-847b71eadaf2" (UID: "22d42849-16e2-4f88-93b8-847b71eadaf2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:58:56.771567 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.771523    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:56.788198 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.788166    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6l7s2\" (UniqueName: \"kubernetes.io/projected/22d42849-16e2-4f88-93b8-847b71eadaf2-kube-api-access-6l7s2\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:58:56.788198 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.788194    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22d42849-16e2-4f88-93b8-847b71eadaf2-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:58:56.940090 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:56.940003    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:57.178626 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.178297    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:57.196340 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.183273    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs" event={"ID":"22d42849-16e2-4f88-93b8-847b71eadaf2","Type":"ContainerDied","Data":"4001a23220ae43c082c6777ae4306aab12d3b787573ca544d155c09d4a5382f2"}
Oct 30 05:58:57.196340 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.183336    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4001a23220ae43c082c6777ae4306aab12d3b787573ca544d155c09d4a5382f2"
Oct 30 05:58:57.196340 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.183415    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"
Oct 30 05:58:57.367573 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.367066    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:57.430975 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.430927    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"]
Oct 30 05:58:57.434057 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.433999    5066 cpu_manager.go:414] "containerMap: removing container" podUID="22d42849-16e2-4f88-93b8-847b71eadaf2" containerName="revoke-keys"
Oct 30 05:58:57.435322 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.435301    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22d42849-16e2-4f88-93b8-847b71eadaf2" containerName="revoke-keys"
Oct 30 05:58:57.435768 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.435743    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="22d42849-16e2-4f88-93b8-847b71eadaf2" containerName="revoke-keys"
Oct 30 05:58:57.440882 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.440614    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:58:57.463656 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.463548    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"]
Oct 30 05:58:57.464280 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.464251    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 05:58:57.464988 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.464960    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 05:58:57.505681 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.500295    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5\" (UID: \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:58:57.505681 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.500545    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcbvw\" (UniqueName: \"kubernetes.io/projected/64cec9a7-97d8-403e-bb81-3cd2ab779b23-kube-api-access-qcbvw\") pod \"maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5\" (UID: \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:58:57.505681 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.500627    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5\" (UID: \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:58:57.603329 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.603297    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcbvw\" (UniqueName: \"kubernetes.io/projected/64cec9a7-97d8-403e-bb81-3cd2ab779b23-kube-api-access-qcbvw\") pod \"maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5\" (UID: \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:58:57.603571 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.603348    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5\" (UID: \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:58:57.603768 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.603681    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5\" (UID: \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:58:57.605220 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.605182    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5\" (UID: \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:58:57.606932 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.606906    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5\" (UID: \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:58:57.630111 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.630080    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcbvw\" (UniqueName: \"kubernetes.io/projected/64cec9a7-97d8-403e-bb81-3cd2ab779b23-kube-api-access-qcbvw\") pod \"maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5\" (UID: \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:58:57.824359 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.824264    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:58:57.996718 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:57.996680    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:58.176011 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:58.175304    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"]
Oct 30 05:58:58.179839 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:58.179807    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:58.184716 ip-10-0-136-173 kubenswrapper[5066]: W1030 05:58:58.184684    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64cec9a7_97d8_403e_bb81_3cd2ab779b23.slice/crio-c44bf5ed232ede5356d4afd414846cd5322dd0bf765a1dbdc6868501e5287ab7 WatchSource:0}: Error finding container c44bf5ed232ede5356d4afd414846cd5322dd0bf765a1dbdc6868501e5287ab7: Status 404 returned error can't find the container with id c44bf5ed232ede5356d4afd414846cd5322dd0bf765a1dbdc6868501e5287ab7
Oct 30 05:58:58.736690 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:58.729078    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:58:59.203776 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:59.203744    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5" event={"ID":"64cec9a7-97d8-403e-bb81-3cd2ab779b23","Type":"ContainerStarted","Data":"d50ef2c84a6c7b65354028a8a29098a4086f9eeee115b91b2b7595aebe5e5130"}
Oct 30 05:58:59.203988 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:59.203802    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5" event={"ID":"64cec9a7-97d8-403e-bb81-3cd2ab779b23","Type":"ContainerStarted","Data":"c44bf5ed232ede5356d4afd414846cd5322dd0bf765a1dbdc6868501e5287ab7"}
Oct 30 05:58:59.261908 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:59.261847    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5" podStartSLOduration=2.261825598 podStartE2EDuration="2.261825598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 05:58:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 05:58:59.249506733 +0000 UTC m=+1642.994792066" watchObservedRunningTime="2026-10-30 05:58:59.261825598 +0000 UTC m=+1643.007110933"
Oct 30 05:58:59.272687 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:58:59.272055    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:00.090612 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:00.088314    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:00.228675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:00.228639    5066 generic.go:356] "Generic (PLEG): container finished" podID="64cec9a7-97d8-403e-bb81-3cd2ab779b23" containerID="d50ef2c84a6c7b65354028a8a29098a4086f9eeee115b91b2b7595aebe5e5130" exitCode=0
Oct 30 05:59:00.228675 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:00.228659    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5" event={"ID":"64cec9a7-97d8-403e-bb81-3cd2ab779b23","Type":"ContainerDied","Data":"d50ef2c84a6c7b65354028a8a29098a4086f9eeee115b91b2b7595aebe5e5130"}
Oct 30 05:59:00.753487 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:00.753275    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:00.980436 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:00.980381    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"]
Oct 30 05:59:01.009678 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.007807    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b2243ef-4jwf7"]
Oct 30 05:59:01.051772 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.051741    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:01.237321 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.237174    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:01.512938 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.512910    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:59:01.583363 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.583270    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:01.701401 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.701368    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-openshift-ca\") pod \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\" (UID: \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\") "
Oct 30 05:59:01.701584 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.701492    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-xks-ca\") pod \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\" (UID: \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\") "
Oct 30 05:59:01.701765 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.701743    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-openshift-ca" pod "64cec9a7-97d8-403e-bb81-3cd2ab779b23" (UID: "64cec9a7-97d8-403e-bb81-3cd2ab779b23"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:59:01.701816 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.701777    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64cec9a7-97d8-403e-bb81-3cd2ab779b23-kube-api-access-qcbvw\" (UniqueName: \"kubernetes.io/projected/64cec9a7-97d8-403e-bb81-3cd2ab779b23-kube-api-access-qcbvw\") pod \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\" (UID: \"64cec9a7-97d8-403e-bb81-3cd2ab779b23\") "
Oct 30 05:59:01.702403 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.702381    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:59:01.703802 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.703783    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-xks-ca" pod "64cec9a7-97d8-403e-bb81-3cd2ab779b23" (UID: "64cec9a7-97d8-403e-bb81-3cd2ab779b23"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 05:59:01.704086 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.704071    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64cec9a7-97d8-403e-bb81-3cd2ab779b23-kube-api-access-qcbvw" pod "64cec9a7-97d8-403e-bb81-3cd2ab779b23" (UID: "64cec9a7-97d8-403e-bb81-3cd2ab779b23"). InnerVolumeSpecName "kube-api-access-qcbvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:59:01.803730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.803700    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcbvw\" (UniqueName: \"kubernetes.io/projected/64cec9a7-97d8-403e-bb81-3cd2ab779b23-kube-api-access-qcbvw\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:59:01.803730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:01.803726    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/64cec9a7-97d8-403e-bb81-3cd2ab779b23-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:59:02.247525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:02.247497    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"
Oct 30 05:59:02.247525 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:02.247518    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5" event={"ID":"64cec9a7-97d8-403e-bb81-3cd2ab779b23","Type":"ContainerDied","Data":"c44bf5ed232ede5356d4afd414846cd5322dd0bf765a1dbdc6868501e5287ab7"}
Oct 30 05:59:02.248038 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:02.247567    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c44bf5ed232ede5356d4afd414846cd5322dd0bf765a1dbdc6868501e5287ab7"
Oct 30 05:59:02.408166 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:02.408123    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a87c9be5-b7ef-4639-9551-c4edd7913d04" path="/var/lib/kubelet/pods/a87c9be5-b7ef-4639-9551-c4edd7913d04/volumes"
Oct 30 05:59:02.562827 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:59:02.562741    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:59:03.181538 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:03.181508    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:03.755070 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:03.754822    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:03.945658 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:03.942060    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:05.460002 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:05.459967    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:05.676118 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:05.676093    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:05.848663 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:05.846924    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:06.601917 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:06.601886    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:07.155534 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:07.155501    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"]
Oct 30 05:59:07.155828 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:07.155803    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerName="istio-proxy" containerID="cri-o://c6f4f8cb9e4e359b620c1e7cce75f393ee594fc2cda1eff3e225ba642035bc30" gracePeriod=30
Oct 30 05:59:08.032137 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:08.032105    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:08.200090 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:08.200056    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:09.848619 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:09.848320    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:11.412271 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:11.412211    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:12.335492 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.335389    5066 generic.go:356] "Generic (PLEG): container finished" podID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerID="c6f4f8cb9e4e359b620c1e7cce75f393ee594fc2cda1eff3e225ba642035bc30" exitCode=0
Oct 30 05:59:12.335492 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.335425    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" event={"ID":"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877","Type":"ContainerDied","Data":"c6f4f8cb9e4e359b620c1e7cce75f393ee594fc2cda1eff3e225ba642035bc30"}
Oct 30 05:59:12.485000 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.484972    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:59:12.530410 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.530374    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-kube-api-access-98tth\" (UniqueName: \"kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-kube-api-access-98tth\") pod \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") "
Oct 30 05:59:12.530614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.530434    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-socket\") pod \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") "
Oct 30 05:59:12.530614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.530467    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-credential-socket\") pod \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") "
Oct 30 05:59:12.530614 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.530547    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-podinfo\") pod \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") "
Oct 30 05:59:12.530796 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.530641    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-envoy\") pod \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") "
Oct 30 05:59:12.530796 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.530706    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-data\") pod \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") "
Oct 30 05:59:12.530796 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.530791    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-certs\") pod \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") "
Oct 30 05:59:12.530954 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.530816    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-socket" pod "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" (UID: "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:59:12.530954 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.530907    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istiod-ca-cert\") pod \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") "
Oct 30 05:59:12.531056 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.530967    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-token\" (UniqueName: \"kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-token\") pod \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\" (UID: \"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877\") "
Oct 30 05:59:12.531135 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.530904    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-credential-socket" pod "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" (UID: "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:59:12.531370 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.530993    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-data" pod "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" (UID: "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:59:12.531370 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.531096    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-certs" pod "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" (UID: "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:59:12.531370 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.531248    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istiod-ca-cert" pod "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" (UID: "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 05:59:12.531951 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.531924    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:59:12.532096 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.531954    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:59:12.532096 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.531969    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:59:12.532096 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.531982    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:59:12.532096 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.531995    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:59:12.533103 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.533072    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-podinfo" pod "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" (UID: "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 05:59:12.533432 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.533408    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-envoy" pod "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" (UID: "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 05:59:12.533630 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.533606    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-kube-api-access-98tth" pod "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" (UID: "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877"). InnerVolumeSpecName "kube-api-access-98tth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:59:12.533812 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.533792    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-token" pod "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" (UID: "af5bf1c5-1f01-4cb6-9cb6-21a4516f8877"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 05:59:12.633730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.633700    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98tth\" (UniqueName: \"kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-kube-api-access-98tth\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:59:12.633730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.633728    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:59:12.633730 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.633738    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:59:12.633963 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:12.633746    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 05:59:13.021281 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:13.021239    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:13.242616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:13.242552    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:13.358581 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:13.358010    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"
Oct 30 05:59:13.358581 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:13.358154    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc" event={"ID":"af5bf1c5-1f01-4cb6-9cb6-21a4516f8877","Type":"ContainerDied","Data":"6d8d6066c275ed7e31d03e853db3799f5b76c74214512e5bd75cd81a8515eaa3"}
Oct 30 05:59:13.358581 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:13.358217    5066 scope.go:122] "RemoveContainer" containerID="c6f4f8cb9e4e359b620c1e7cce75f393ee594fc2cda1eff3e225ba642035bc30"
Oct 30 05:59:13.420347 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:13.420318    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"]
Oct 30 05:59:13.434118 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:13.434092    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-9fgqc"]
Oct 30 05:59:14.403993 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:14.403961    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" path="/var/lib/kubelet/pods/af5bf1c5-1f01-4cb6-9cb6-21a4516f8877/volumes"
Oct 30 05:59:14.803224 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:14.802364    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:16.501697 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:16.501551    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:18.221083 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:18.221054    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:19.838442 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:19.838381    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:21.546790 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:21.546715    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:23.328486 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:23.328420    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:25.012669 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:25.012323    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:26.545303 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:26.545243    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:28.194309 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:28.194229    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:29.433093 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:29.433066    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:29.826984 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:29.826895    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:31.312026 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:31.311993    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:31.478929 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:31.478897    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:32.418617 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:32.417885    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:32.589104 ip-10-0-136-173 kubenswrapper[5066]: E1030 05:59:32.589060    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 05:59:33.137094 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:33.137059    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:34.822812 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:34.822775    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:36.394615 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:36.393693    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:38.114194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:38.114162    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:38.808719 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:38.808687    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:39.778959 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:39.778928    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:41.248439 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:41.245709    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:43.001616 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:42.996052    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:43.921045 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:43.921013    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:44.768130 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:44.768065    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:45.212872 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:45.212826    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:46.335721 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:46.335660    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:48.045233 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:48.045186    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:49.532181 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:49.532130    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:49.709839 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:49.709806    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:50.153644 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:50.153608    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:50.471453 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:50.471344    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:51.422265 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:51.422233    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:52.116224 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:52.116190    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:52.305884 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:52.305849    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:52.536533 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:52.536499    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:53.202524 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:53.202483    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:53.872812 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:53.872782    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:53.984194 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:53.984162    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"]
Oct 30 05:59:53.998089 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:53.998059    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b202a0-gpsrs"]
Oct 30 05:59:54.140875 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:54.140764    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:54.335835 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:54.335792    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:54.404155 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:54.404130    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22d42849-16e2-4f88-93b8-847b71eadaf2" path="/var/lib/kubelet/pods/22d42849-16e2-4f88-93b8-847b71eadaf2/volumes"
Oct 30 05:59:54.529199 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:54.529170    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:54.781204 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:54.781118    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:56.017602 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:56.017564    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:56.213041 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:56.213008    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:56.407108 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:56.407040    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:57.960624 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:57.960549    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 05:59:59.736365 ip-10-0-136-173 kubenswrapper[5066]: I1030 05:59:59.736323    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:00.156036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.156008    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"]
Oct 30 06:00:00.156698 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.156684    5066 cpu_manager.go:414] "containerMap: removing container" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerName="istio-proxy"
Oct 30 06:00:00.156758 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.156699    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerName="istio-proxy"
Oct 30 06:00:00.156758 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.156717    5066 cpu_manager.go:414] "containerMap: removing container" podUID="64cec9a7-97d8-403e-bb81-3cd2ab779b23" containerName="revoke-keys"
Oct 30 06:00:00.156758 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.156722    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64cec9a7-97d8-403e-bb81-3cd2ab779b23" containerName="revoke-keys"
Oct 30 06:00:00.156854 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.156846    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="af5bf1c5-1f01-4cb6-9cb6-21a4516f8877" containerName="istio-proxy"
Oct 30 06:00:00.156892 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.156858    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="64cec9a7-97d8-403e-bb81-3cd2ab779b23" containerName="revoke-keys"
Oct 30 06:00:00.160801 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.160754    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"
Oct 30 06:00:00.165397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.165373    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"]
Oct 30 06:00:00.170507 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.170487    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"
Oct 30 06:00:00.190129 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.190063    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"]
Oct 30 06:00:00.193302 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.193274    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"]
Oct 30 06:00:00.194540 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.194508    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:00:00.201968 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.201941    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"
Oct 30 06:00:00.213879 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.213849    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"]
Oct 30 06:00:00.219978 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.219955    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"]
Oct 30 06:00:00.220132 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.220064    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"
Oct 30 06:00:00.223299 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.223268    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"]
Oct 30 06:00:00.229007 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.228984    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"
Oct 30 06:00:00.237819 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.236207    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"]
Oct 30 06:00:00.245366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.245260    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"]
Oct 30 06:00:00.245493 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.245369    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"
Oct 30 06:00:00.247517 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.247238    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"]
Oct 30 06:00:00.253546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.249666    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4\" (UID: \"314aebbd-980a-49e5-a6e5-8b807c5eb4b8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"
Oct 30 06:00:00.253546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.249708    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb970e05-080d-4383-9003-2380f254d3ec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll\" (UID: \"cb970e05-080d-4383-9003-2380f254d3ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"
Oct 30 06:00:00.253546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.249813    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/263ee938-5a4a-43a9-bee6-41c00c03eacd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd\" (UID: \"263ee938-5a4a-43a9-bee6-41c00c03eacd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"
Oct 30 06:00:00.253546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.249888    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-565hg\" (UniqueName: \"kubernetes.io/projected/263ee938-5a4a-43a9-bee6-41c00c03eacd-kube-api-access-565hg\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd\" (UID: \"263ee938-5a4a-43a9-bee6-41c00c03eacd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"
Oct 30 06:00:00.253546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.249993    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmxdf\" (UniqueName: \"kubernetes.io/projected/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-kube-api-access-wmxdf\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4\" (UID: \"314aebbd-980a-49e5-a6e5-8b807c5eb4b8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"
Oct 30 06:00:00.253546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.250138    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz6x9\" (UniqueName: \"kubernetes.io/projected/926d956d-0651-401b-8661-1cb2bd101f81-kube-api-access-zz6x9\") pod \"maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2\" (UID: \"926d956d-0651-401b-8661-1cb2bd101f81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"
Oct 30 06:00:00.253546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.250264    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/926d956d-0651-401b-8661-1cb2bd101f81-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2\" (UID: \"926d956d-0651-401b-8661-1cb2bd101f81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"
Oct 30 06:00:00.253546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.250315    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvc9z\" (UniqueName: \"kubernetes.io/projected/cb970e05-080d-4383-9003-2380f254d3ec-kube-api-access-wvc9z\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll\" (UID: \"cb970e05-080d-4383-9003-2380f254d3ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"
Oct 30 06:00:00.261359 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.261329    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"]
Oct 30 06:00:00.266527 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.266506    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"]
Oct 30 06:00:00.272177 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.272153    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"]
Oct 30 06:00:00.276915 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.276897    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"
Oct 30 06:00:00.292153 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.292124    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"]
Oct 30 06:00:00.351749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.351714    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/926d956d-0651-401b-8661-1cb2bd101f81-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2\" (UID: \"926d956d-0651-401b-8661-1cb2bd101f81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"
Oct 30 06:00:00.351958 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.351772    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvc9z\" (UniqueName: \"kubernetes.io/projected/cb970e05-080d-4383-9003-2380f254d3ec-kube-api-access-wvc9z\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll\" (UID: \"cb970e05-080d-4383-9003-2380f254d3ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"
Oct 30 06:00:00.351958 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.351808    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvnv4\" (UniqueName: \"kubernetes.io/projected/a0e477b3-de84-4a4d-bedb-f47ea744632a-kube-api-access-xvnv4\") pod \"maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk\" (UID: \"a0e477b3-de84-4a4d-bedb-f47ea744632a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"
Oct 30 06:00:00.351958 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.351869    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p9jr\" (UniqueName: \"kubernetes.io/projected/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-kube-api-access-9p9jr\") pod \"maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj\" (UID: \"ea0227f2-0020-4aa2-b62b-843fcfbeafe1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"
Oct 30 06:00:00.352150 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.352025    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7b36512-f641-4b2a-9cf9-f8deade880bc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw\" (UID: \"a7b36512-f641-4b2a-9cf9-f8deade880bc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"
Oct 30 06:00:00.352210 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.352188    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj\" (UID: \"ea0227f2-0020-4aa2-b62b-843fcfbeafe1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"
Oct 30 06:00:00.352288 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.352265    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4\" (UID: \"314aebbd-980a-49e5-a6e5-8b807c5eb4b8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"
Oct 30 06:00:00.352379 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.352319    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb970e05-080d-4383-9003-2380f254d3ec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll\" (UID: \"cb970e05-080d-4383-9003-2380f254d3ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"
Oct 30 06:00:00.352467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.352438    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/263ee938-5a4a-43a9-bee6-41c00c03eacd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd\" (UID: \"263ee938-5a4a-43a9-bee6-41c00c03eacd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"
Oct 30 06:00:00.352575 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.352523    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-565hg\" (UniqueName: \"kubernetes.io/projected/263ee938-5a4a-43a9-bee6-41c00c03eacd-kube-api-access-565hg\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd\" (UID: \"263ee938-5a4a-43a9-bee6-41c00c03eacd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"
Oct 30 06:00:00.352575 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.352547    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/926d956d-0651-401b-8661-1cb2bd101f81-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2\" (UID: \"926d956d-0651-401b-8661-1cb2bd101f81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"
Oct 30 06:00:00.352723 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.352572    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v64cv\" (UniqueName: \"kubernetes.io/projected/a7b36512-f641-4b2a-9cf9-f8deade880bc-kube-api-access-v64cv\") pod \"maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw\" (UID: \"a7b36512-f641-4b2a-9cf9-f8deade880bc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"
Oct 30 06:00:00.352723 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.352641    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmxdf\" (UniqueName: \"kubernetes.io/projected/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-kube-api-access-wmxdf\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4\" (UID: \"314aebbd-980a-49e5-a6e5-8b807c5eb4b8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"
Oct 30 06:00:00.352723 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.352680    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz6x9\" (UniqueName: \"kubernetes.io/projected/926d956d-0651-401b-8661-1cb2bd101f81-kube-api-access-zz6x9\") pod \"maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2\" (UID: \"926d956d-0651-401b-8661-1cb2bd101f81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"
Oct 30 06:00:00.352841 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.352731    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e477b3-de84-4a4d-bedb-f47ea744632a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk\" (UID: \"a0e477b3-de84-4a4d-bedb-f47ea744632a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"
Oct 30 06:00:00.353128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.353098    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4\" (UID: \"314aebbd-980a-49e5-a6e5-8b807c5eb4b8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"
Oct 30 06:00:00.353182 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.353148    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb970e05-080d-4383-9003-2380f254d3ec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll\" (UID: \"cb970e05-080d-4383-9003-2380f254d3ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"
Oct 30 06:00:00.353182 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.353175    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/263ee938-5a4a-43a9-bee6-41c00c03eacd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd\" (UID: \"263ee938-5a4a-43a9-bee6-41c00c03eacd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"
Oct 30 06:00:00.382371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.382340    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvc9z\" (UniqueName: \"kubernetes.io/projected/cb970e05-080d-4383-9003-2380f254d3ec-kube-api-access-wvc9z\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll\" (UID: \"cb970e05-080d-4383-9003-2380f254d3ec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"
Oct 30 06:00:00.382530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.382421    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz6x9\" (UniqueName: \"kubernetes.io/projected/926d956d-0651-401b-8661-1cb2bd101f81-kube-api-access-zz6x9\") pod \"maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2\" (UID: \"926d956d-0651-401b-8661-1cb2bd101f81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"
Oct 30 06:00:00.382530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.382468    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmxdf\" (UniqueName: \"kubernetes.io/projected/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-kube-api-access-wmxdf\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4\" (UID: \"314aebbd-980a-49e5-a6e5-8b807c5eb4b8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"
Oct 30 06:00:00.382727 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.382535    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-565hg\" (UniqueName: \"kubernetes.io/projected/263ee938-5a4a-43a9-bee6-41c00c03eacd-kube-api-access-565hg\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd\" (UID: \"263ee938-5a4a-43a9-bee6-41c00c03eacd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"
Oct 30 06:00:00.454581 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.454500    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v64cv\" (UniqueName: \"kubernetes.io/projected/a7b36512-f641-4b2a-9cf9-f8deade880bc-kube-api-access-v64cv\") pod \"maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw\" (UID: \"a7b36512-f641-4b2a-9cf9-f8deade880bc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"
Oct 30 06:00:00.454581 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.454564    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e477b3-de84-4a4d-bedb-f47ea744632a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk\" (UID: \"a0e477b3-de84-4a4d-bedb-f47ea744632a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"
Oct 30 06:00:00.454827 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.454650    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvnv4\" (UniqueName: \"kubernetes.io/projected/a0e477b3-de84-4a4d-bedb-f47ea744632a-kube-api-access-xvnv4\") pod \"maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk\" (UID: \"a0e477b3-de84-4a4d-bedb-f47ea744632a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"
Oct 30 06:00:00.454827 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.454702    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p9jr\" (UniqueName: \"kubernetes.io/projected/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-kube-api-access-9p9jr\") pod \"maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj\" (UID: \"ea0227f2-0020-4aa2-b62b-843fcfbeafe1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"
Oct 30 06:00:00.454827 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.454791    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7b36512-f641-4b2a-9cf9-f8deade880bc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw\" (UID: \"a7b36512-f641-4b2a-9cf9-f8deade880bc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"
Oct 30 06:00:00.455006 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.454863    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj\" (UID: \"ea0227f2-0020-4aa2-b62b-843fcfbeafe1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"
Oct 30 06:00:00.455554 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.455519    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e477b3-de84-4a4d-bedb-f47ea744632a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk\" (UID: \"a0e477b3-de84-4a4d-bedb-f47ea744632a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"
Oct 30 06:00:00.455778 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.455748    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7b36512-f641-4b2a-9cf9-f8deade880bc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw\" (UID: \"a7b36512-f641-4b2a-9cf9-f8deade880bc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"
Oct 30 06:00:00.455898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.455797    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj\" (UID: \"ea0227f2-0020-4aa2-b62b-843fcfbeafe1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"
Oct 30 06:00:00.477265 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.477237    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"
Oct 30 06:00:00.480717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.480689    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v64cv\" (UniqueName: \"kubernetes.io/projected/a7b36512-f641-4b2a-9cf9-f8deade880bc-kube-api-access-v64cv\") pod \"maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw\" (UID: \"a7b36512-f641-4b2a-9cf9-f8deade880bc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"
Oct 30 06:00:00.480717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.480707    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvnv4\" (UniqueName: \"kubernetes.io/projected/a0e477b3-de84-4a4d-bedb-f47ea744632a-kube-api-access-xvnv4\") pod \"maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk\" (UID: \"a0e477b3-de84-4a4d-bedb-f47ea744632a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"
Oct 30 06:00:00.481001 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.480980    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p9jr\" (UniqueName: \"kubernetes.io/projected/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-kube-api-access-9p9jr\") pod \"maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj\" (UID: \"ea0227f2-0020-4aa2-b62b-843fcfbeafe1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"
Oct 30 06:00:00.495788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.495764    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"
Oct 30 06:00:00.523312 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.523282    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"
Oct 30 06:00:00.542354 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.542298    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"
Oct 30 06:00:00.565404 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.565156    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"
Oct 30 06:00:00.581712 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.581680    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"
Oct 30 06:00:00.597973 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.597717    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"
Oct 30 06:00:00.805903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.805860    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"]
Oct 30 06:00:00.817019 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:00:00.816277    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod314aebbd_980a_49e5_a6e5_8b807c5eb4b8.slice/crio-6075a4e4e13367a94518202f4a1d4ae49c946bdf33455d475532d85b3a582039 WatchSource:0}: Error finding container 6075a4e4e13367a94518202f4a1d4ae49c946bdf33455d475532d85b3a582039: Status 404 returned error can't find the container with id 6075a4e4e13367a94518202f4a1d4ae49c946bdf33455d475532d85b3a582039
Oct 30 06:00:00.835115 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.835084    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"]
Oct 30 06:00:00.836552 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:00:00.836523    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926d956d_0651_401b_8661_1cb2bd101f81.slice/crio-83bf5a64e61640574b5a1edb31717ced718741d67628c64af500882714fe977f WatchSource:0}: Error finding container 83bf5a64e61640574b5a1edb31717ced718741d67628c64af500882714fe977f: Status 404 returned error can't find the container with id 83bf5a64e61640574b5a1edb31717ced718741d67628c64af500882714fe977f
Oct 30 06:00:00.880956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.880931    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"]
Oct 30 06:00:00.987674 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.987615    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"]
Oct 30 06:00:00.995273 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.995250    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"]
Oct 30 06:00:00.999549 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:00.999523    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"]
Oct 30 06:00:01.028829 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:00:01.028799    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b36512_f641_4b2a_9cf9_f8deade880bc.slice/crio-b19ce4832c11a322d7ee25af3f15c2bb04405b82c2cee3bed0755c0e6687c9f3 WatchSource:0}: Error finding container b19ce4832c11a322d7ee25af3f15c2bb04405b82c2cee3bed0755c0e6687c9f3: Status 404 returned error can't find the container with id b19ce4832c11a322d7ee25af3f15c2bb04405b82c2cee3bed0755c0e6687c9f3
Oct 30 06:00:01.030570 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:00:01.030547    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod263ee938_5a4a_43a9_bee6_41c00c03eacd.slice/crio-4be6a2f05add32c8e9effa33b912b522126f06f4a3dd42ce9e64ada9de5e0e74 WatchSource:0}: Error finding container 4be6a2f05add32c8e9effa33b912b522126f06f4a3dd42ce9e64ada9de5e0e74: Status 404 returned error can't find the container with id 4be6a2f05add32c8e9effa33b912b522126f06f4a3dd42ce9e64ada9de5e0e74
Oct 30 06:00:01.038779 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:00:01.038739    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea0227f2_0020_4aa2_b62b_843fcfbeafe1.slice/crio-54eccb411e92d2ca35023d58f32b459377ea6d2eea737c29d5edcee5b9dad453 WatchSource:0}: Error finding container 54eccb411e92d2ca35023d58f32b459377ea6d2eea737c29d5edcee5b9dad453: Status 404 returned error can't find the container with id 54eccb411e92d2ca35023d58f32b459377ea6d2eea737c29d5edcee5b9dad453
Oct 30 06:00:01.187083 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.185715    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"]
Oct 30 06:00:01.304238 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.304199    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"]
Oct 30 06:00:01.330130 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.330104    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:01.962713 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.959814    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj" event={"ID":"ea0227f2-0020-4aa2-b62b-843fcfbeafe1","Type":"ContainerStarted","Data":"54eccb411e92d2ca35023d58f32b459377ea6d2eea737c29d5edcee5b9dad453"}
Oct 30 06:00:01.962713 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.960005    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj" podUID="ea0227f2-0020-4aa2-b62b-843fcfbeafe1" containerName="cleanup" containerID="cri-o://050282ee377a47ae27cfc32f8ce08f06396c0167120dc1eed987eccb0744057e" gracePeriod=30
Oct 30 06:00:01.964838 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.964076    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw" event={"ID":"a7b36512-f641-4b2a-9cf9-f8deade880bc","Type":"ContainerStarted","Data":"b19ce4832c11a322d7ee25af3f15c2bb04405b82c2cee3bed0755c0e6687c9f3"}
Oct 30 06:00:01.970399 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.970375    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll" event={"ID":"cb970e05-080d-4383-9003-2380f254d3ec","Type":"ContainerStarted","Data":"646894c6b04f965dc49d8232d0311a2ff272970bc502ace9dcf44021122580fb"}
Oct 30 06:00:01.970470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.970433    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll" event={"ID":"cb970e05-080d-4383-9003-2380f254d3ec","Type":"ContainerStarted","Data":"deae1352ce81481c9622e68fb79c5379c9b35b605796f69f722b0c911e03b730"}
Oct 30 06:00:01.973434 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.973405    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk" event={"ID":"a0e477b3-de84-4a4d-bedb-f47ea744632a","Type":"ContainerStarted","Data":"49b7f706727813e0f177d1715dc5f6927790bb595575500b2ce2665777642cfb"}
Oct 30 06:00:01.975848 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.975760    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4" event={"ID":"314aebbd-980a-49e5-a6e5-8b807c5eb4b8","Type":"ContainerStarted","Data":"f313578bfde0307885e9cd905e283cde7f38c7415cdfa52b2e3a25b5062a313e"}
Oct 30 06:00:01.976051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.976032    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4" event={"ID":"314aebbd-980a-49e5-a6e5-8b807c5eb4b8","Type":"ContainerStarted","Data":"6075a4e4e13367a94518202f4a1d4ae49c946bdf33455d475532d85b3a582039"}
Oct 30 06:00:01.978203 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.977910    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd" event={"ID":"263ee938-5a4a-43a9-bee6-41c00c03eacd","Type":"ContainerStarted","Data":"4be6a2f05add32c8e9effa33b912b522126f06f4a3dd42ce9e64ada9de5e0e74"}
Oct 30 06:00:01.980544 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.979898    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2" event={"ID":"926d956d-0651-401b-8661-1cb2bd101f81","Type":"ContainerStarted","Data":"eb3048cad8e7458e0ce677b156cd7b53513a987ca5c5b9b0ae9a160375c8a3f1"}
Oct 30 06:00:01.980544 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.979935    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2" event={"ID":"926d956d-0651-401b-8661-1cb2bd101f81","Type":"ContainerStarted","Data":"83bf5a64e61640574b5a1edb31717ced718741d67628c64af500882714fe977f"}
Oct 30 06:00:01.980544 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:01.980225    5066 scope.go:122] "RemoveContainer" containerID="eb3048cad8e7458e0ce677b156cd7b53513a987ca5c5b9b0ae9a160375c8a3f1"
Oct 30 06:00:02.006347 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.006287    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:02.023388 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.023341    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj" podStartSLOduration=2.023328426 podStartE2EDuration="2.023328426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:00:02.014255699 +0000 UTC m=+1705.759541056" watchObservedRunningTime="2026-10-30 06:00:02.023328426 +0000 UTC m=+1705.768613757"
Oct 30 06:00:02.072453 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.072393    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll" podStartSLOduration=2.072373876 podStartE2EDuration="2.072373876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:00:02.069453601 +0000 UTC m=+1705.814738932" watchObservedRunningTime="2026-10-30 06:00:02.072373876 +0000 UTC m=+1705.817659208"
Oct 30 06:00:02.171976 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.171914    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4" podStartSLOduration=2.171893886 podStartE2EDuration="2.171893886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:00:02.170554171 +0000 UTC m=+1705.915839504" watchObservedRunningTime="2026-10-30 06:00:02.171893886 +0000 UTC m=+1705.917179220"
Oct 30 06:00:02.249099 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.249038    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd" podStartSLOduration=2.249019582 podStartE2EDuration="2.249019582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:00:02.241102637 +0000 UTC m=+1705.986387969" watchObservedRunningTime="2026-10-30 06:00:02.249019582 +0000 UTC m=+1705.994304916"
Oct 30 06:00:02.330893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.330831    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw" podStartSLOduration=2.330812205 podStartE2EDuration="2.330812205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:00:02.328669983 +0000 UTC m=+1706.073955318" watchObservedRunningTime="2026-10-30 06:00:02.330812205 +0000 UTC m=+1706.076097539"
Oct 30 06:00:02.331084 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.330862    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:02.341898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.341868    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"]
Oct 30 06:00:02.345912 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.345884    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4fc75f-s4pg5"]
Oct 30 06:00:02.410572 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.410540    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64cec9a7-97d8-403e-bb81-3cd2ab779b23" path="/var/lib/kubelet/pods/64cec9a7-97d8-403e-bb81-3cd2ab779b23/volumes"
Oct 30 06:00:02.558381 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.558350    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:02.599636 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:02.599602    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:00:02.657711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.657683    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"
Oct 30 06:00:02.745004 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.744965    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-maas-api-service-ca\") pod \"ea0227f2-0020-4aa2-b62b-843fcfbeafe1\" (UID: \"ea0227f2-0020-4aa2-b62b-843fcfbeafe1\") "
Oct 30 06:00:02.745395 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.745370    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-kube-api-access-9p9jr\" (UniqueName: \"kubernetes.io/projected/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-kube-api-access-9p9jr\") pod \"ea0227f2-0020-4aa2-b62b-843fcfbeafe1\" (UID: \"ea0227f2-0020-4aa2-b62b-843fcfbeafe1\") "
Oct 30 06:00:02.746653 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.746238    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-maas-api-service-ca" pod "ea0227f2-0020-4aa2-b62b-843fcfbeafe1" (UID: "ea0227f2-0020-4aa2-b62b-843fcfbeafe1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:00:02.749515 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.749490    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-kube-api-access-9p9jr" pod "ea0227f2-0020-4aa2-b62b-843fcfbeafe1" (UID: "ea0227f2-0020-4aa2-b62b-843fcfbeafe1"). InnerVolumeSpecName "kube-api-access-9p9jr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:02.755366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.755342    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:02.867672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.859024    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-maas-api-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:02.867672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.859064    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p9jr\" (UniqueName: \"kubernetes.io/projected/ea0227f2-0020-4aa2-b62b-843fcfbeafe1-kube-api-access-9p9jr\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:02.933879 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:02.933798    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:03.031609 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.028558    5066 generic.go:356] "Generic (PLEG): container finished" podID="ea0227f2-0020-4aa2-b62b-843fcfbeafe1" containerID="050282ee377a47ae27cfc32f8ce08f06396c0167120dc1eed987eccb0744057e" exitCode=6
Oct 30 06:00:03.031609 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.028698    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj" event={"ID":"ea0227f2-0020-4aa2-b62b-843fcfbeafe1","Type":"ContainerDied","Data":"050282ee377a47ae27cfc32f8ce08f06396c0167120dc1eed987eccb0744057e"}
Oct 30 06:00:03.031609 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.028750    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj" event={"ID":"ea0227f2-0020-4aa2-b62b-843fcfbeafe1","Type":"ContainerDied","Data":"54eccb411e92d2ca35023d58f32b459377ea6d2eea737c29d5edcee5b9dad453"}
Oct 30 06:00:03.031609 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.028780    5066 scope.go:122] "RemoveContainer" containerID="050282ee377a47ae27cfc32f8ce08f06396c0167120dc1eed987eccb0744057e"
Oct 30 06:00:03.031609 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.028925    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"
Oct 30 06:00:03.066786 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.061647    5066 generic.go:356] "Generic (PLEG): container finished" podID="a7b36512-f641-4b2a-9cf9-f8deade880bc" containerID="a0b68cd6ba2692079549800b8d78fcdf4afec7ca585afea271c37d4ef2a2f1fc" exitCode=60
Oct 30 06:00:03.066786 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.061798    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw" event={"ID":"a7b36512-f641-4b2a-9cf9-f8deade880bc","Type":"ContainerDied","Data":"a0b68cd6ba2692079549800b8d78fcdf4afec7ca585afea271c37d4ef2a2f1fc"}
Oct 30 06:00:03.066786 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.062395    5066 scope.go:122] "RemoveContainer" containerID="a0b68cd6ba2692079549800b8d78fcdf4afec7ca585afea271c37d4ef2a2f1fc"
Oct 30 06:00:03.082672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.076209    5066 generic.go:356] "Generic (PLEG): container finished" podID="cb970e05-080d-4383-9003-2380f254d3ec" containerID="646894c6b04f965dc49d8232d0311a2ff272970bc502ace9dcf44021122580fb" exitCode=60
Oct 30 06:00:03.082672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.076313    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll" event={"ID":"cb970e05-080d-4383-9003-2380f254d3ec","Type":"ContainerDied","Data":"646894c6b04f965dc49d8232d0311a2ff272970bc502ace9dcf44021122580fb"}
Oct 30 06:00:03.085738 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.085478    5066 scope.go:122] "RemoveContainer" containerID="646894c6b04f965dc49d8232d0311a2ff272970bc502ace9dcf44021122580fb"
Oct 30 06:00:03.086959 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.086460    5066 generic.go:356] "Generic (PLEG): container finished" podID="a0e477b3-de84-4a4d-bedb-f47ea744632a" containerID="241a8209de5a4e6348b542bb59974386e42d6cb889b6cddd661096ee5c07c9e2" exitCode=60
Oct 30 06:00:03.087370 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.087351    5066 scope.go:122] "RemoveContainer" containerID="241a8209de5a4e6348b542bb59974386e42d6cb889b6cddd661096ee5c07c9e2"
Oct 30 06:00:03.089870 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.089792    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk" event={"ID":"a0e477b3-de84-4a4d-bedb-f47ea744632a","Type":"ContainerDied","Data":"241a8209de5a4e6348b542bb59974386e42d6cb889b6cddd661096ee5c07c9e2"}
Oct 30 06:00:03.140924 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.140889    5066 scope.go:122] "RemoveContainer" containerID="050282ee377a47ae27cfc32f8ce08f06396c0167120dc1eed987eccb0744057e"
Oct 30 06:00:03.143157 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:03.143119    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"050282ee377a47ae27cfc32f8ce08f06396c0167120dc1eed987eccb0744057e\": container with ID starting with 050282ee377a47ae27cfc32f8ce08f06396c0167120dc1eed987eccb0744057e not found: ID does not exist" containerID="050282ee377a47ae27cfc32f8ce08f06396c0167120dc1eed987eccb0744057e"
Oct 30 06:00:03.143318 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.143167    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050282ee377a47ae27cfc32f8ce08f06396c0167120dc1eed987eccb0744057e"} err="failed to get container status \"050282ee377a47ae27cfc32f8ce08f06396c0167120dc1eed987eccb0744057e\": rpc error: code = NotFound desc = could not find container \"050282ee377a47ae27cfc32f8ce08f06396c0167120dc1eed987eccb0744057e\": container with ID starting with 050282ee377a47ae27cfc32f8ce08f06396c0167120dc1eed987eccb0744057e not found: ID does not exist"
Oct 30 06:00:03.144477 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.144303    5066 generic.go:356] "Generic (PLEG): container finished" podID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" containerID="f313578bfde0307885e9cd905e283cde7f38c7415cdfa52b2e3a25b5062a313e" exitCode=60
Oct 30 06:00:03.144477 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.144422    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4" event={"ID":"314aebbd-980a-49e5-a6e5-8b807c5eb4b8","Type":"ContainerDied","Data":"f313578bfde0307885e9cd905e283cde7f38c7415cdfa52b2e3a25b5062a313e"}
Oct 30 06:00:03.145125 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.144970    5066 scope.go:122] "RemoveContainer" containerID="f313578bfde0307885e9cd905e283cde7f38c7415cdfa52b2e3a25b5062a313e"
Oct 30 06:00:03.149830 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.148821    5066 generic.go:356] "Generic (PLEG): container finished" podID="263ee938-5a4a-43a9-bee6-41c00c03eacd" containerID="730358c033f803558a7cb372ce0c9b3f78e8860d983ef22658f453d1b0c66d5a" exitCode=60
Oct 30 06:00:03.149830 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.148922    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd" event={"ID":"263ee938-5a4a-43a9-bee6-41c00c03eacd","Type":"ContainerDied","Data":"730358c033f803558a7cb372ce0c9b3f78e8860d983ef22658f453d1b0c66d5a"}
Oct 30 06:00:03.149830 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.149365    5066 scope.go:122] "RemoveContainer" containerID="730358c033f803558a7cb372ce0c9b3f78e8860d983ef22658f453d1b0c66d5a"
Oct 30 06:00:03.150981 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.150935    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"]
Oct 30 06:00:03.154730 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.153391    5066 generic.go:356] "Generic (PLEG): container finished" podID="926d956d-0651-401b-8661-1cb2bd101f81" containerID="eb3048cad8e7458e0ce677b156cd7b53513a987ca5c5b9b0ae9a160375c8a3f1" exitCode=60
Oct 30 06:00:03.154730 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.153439    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2" event={"ID":"926d956d-0651-401b-8661-1cb2bd101f81","Type":"ContainerDied","Data":"eb3048cad8e7458e0ce677b156cd7b53513a987ca5c5b9b0ae9a160375c8a3f1"}
Oct 30 06:00:03.154730 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.153475    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2" event={"ID":"926d956d-0651-401b-8661-1cb2bd101f81","Type":"ContainerStarted","Data":"afae60ac5f523beafa3b357d7c59e0672f0d6baa457224d4bb070886c7d83387"}
Oct 30 06:00:03.157780 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.157753    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-4fc75f-29889000-ws6wj"]
Oct 30 06:00:03.350178 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.348968    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:03.580813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:03.580759    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2" podStartSLOduration=3.580746307 podStartE2EDuration="3.580746307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:00:03.577662312 +0000 UTC m=+1707.322947655" watchObservedRunningTime="2026-10-30 06:00:03.580746307 +0000 UTC m=+1707.326031638"
Oct 30 06:00:04.087694 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.087660    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:04.207669 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.194904    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd" event={"ID":"263ee938-5a4a-43a9-bee6-41c00c03eacd","Type":"ContainerStarted","Data":"3548d8ea450b5d066636519740a5477e2e483f2cf65d9186ffcfad03ab6a8b77"}
Oct 30 06:00:04.230624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.218536    5066 generic.go:356] "Generic (PLEG): container finished" podID="926d956d-0651-401b-8661-1cb2bd101f81" containerID="afae60ac5f523beafa3b357d7c59e0672f0d6baa457224d4bb070886c7d83387" exitCode=60
Oct 30 06:00:04.230624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.218726    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2" event={"ID":"926d956d-0651-401b-8661-1cb2bd101f81","Type":"ContainerDied","Data":"afae60ac5f523beafa3b357d7c59e0672f0d6baa457224d4bb070886c7d83387"}
Oct 30 06:00:04.230624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.218786    5066 scope.go:122] "RemoveContainer" containerID="eb3048cad8e7458e0ce677b156cd7b53513a987ca5c5b9b0ae9a160375c8a3f1"
Oct 30 06:00:04.230624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.219314    5066 scope.go:122] "RemoveContainer" containerID="afae60ac5f523beafa3b357d7c59e0672f0d6baa457224d4bb070886c7d83387"
Oct 30 06:00:04.230624 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:04.219825    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2_odh-ai-gateway-infra(926d956d-0651-401b-8661-1cb2bd101f81)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2" podUID="926d956d-0651-401b-8661-1cb2bd101f81"
Oct 30 06:00:04.230624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.230350    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw" event={"ID":"a7b36512-f641-4b2a-9cf9-f8deade880bc","Type":"ContainerStarted","Data":"78e34d28c8e8da09a96c82bb517944df67c12f69990aff043b280f57f8caaa0b"}
Oct 30 06:00:04.235615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.235069    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll" event={"ID":"cb970e05-080d-4383-9003-2380f254d3ec","Type":"ContainerStarted","Data":"5ca19e1f09b57be5c3d2c7b841235f90d16d02a3f879e2b582d922376760401e"}
Oct 30 06:00:04.250622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.244848    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk" event={"ID":"a0e477b3-de84-4a4d-bedb-f47ea744632a","Type":"ContainerStarted","Data":"71fbb47282c4829bdbdb54d064f3fedfcebc0c7caac1f68c661db3cfe5bd0dbd"}
Oct 30 06:00:04.250622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.249469    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4" event={"ID":"314aebbd-980a-49e5-a6e5-8b807c5eb4b8","Type":"ContainerStarted","Data":"495aa3b0943f3c457495fee3a9482f2248a692fe6c008b7c5a1058e8f01c0cf9"}
Oct 30 06:00:04.415691 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.415659    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea0227f2-0020-4aa2-b62b-843fcfbeafe1" path="/var/lib/kubelet/pods/ea0227f2-0020-4aa2-b62b-843fcfbeafe1/volumes"
Oct 30 06:00:04.448789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.448725    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk" podStartSLOduration=4.448706835 podStartE2EDuration="4.448706835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:00:04.447243581 +0000 UTC m=+1708.192528914" watchObservedRunningTime="2026-10-30 06:00:04.448706835 +0000 UTC m=+1708.193992168"
Oct 30 06:00:04.851383 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:04.850579    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:05.229703 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.224700    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:05.341616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.331850    5066 scope.go:122] "RemoveContainer" containerID="afae60ac5f523beafa3b357d7c59e0672f0d6baa457224d4bb070886c7d83387"
Oct 30 06:00:05.345616 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:05.345316    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2_odh-ai-gateway-infra(926d956d-0651-401b-8661-1cb2bd101f81)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2" podUID="926d956d-0651-401b-8661-1cb2bd101f81"
Oct 30 06:00:05.366617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.360299    5066 generic.go:356] "Generic (PLEG): container finished" podID="a7b36512-f641-4b2a-9cf9-f8deade880bc" containerID="78e34d28c8e8da09a96c82bb517944df67c12f69990aff043b280f57f8caaa0b" exitCode=60
Oct 30 06:00:05.366617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.360414    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw" event={"ID":"a7b36512-f641-4b2a-9cf9-f8deade880bc","Type":"ContainerDied","Data":"78e34d28c8e8da09a96c82bb517944df67c12f69990aff043b280f57f8caaa0b"}
Oct 30 06:00:05.366617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.360471    5066 scope.go:122] "RemoveContainer" containerID="a0b68cd6ba2692079549800b8d78fcdf4afec7ca585afea271c37d4ef2a2f1fc"
Oct 30 06:00:05.366617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.361153    5066 scope.go:122] "RemoveContainer" containerID="78e34d28c8e8da09a96c82bb517944df67c12f69990aff043b280f57f8caaa0b"
Oct 30 06:00:05.366617 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:05.361668    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw_odh-ai-gateway-infra(a7b36512-f641-4b2a-9cf9-f8deade880bc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc"
Oct 30 06:00:05.399658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.396249    5066 generic.go:356] "Generic (PLEG): container finished" podID="cb970e05-080d-4383-9003-2380f254d3ec" containerID="5ca19e1f09b57be5c3d2c7b841235f90d16d02a3f879e2b582d922376760401e" exitCode=60
Oct 30 06:00:05.399658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.396407    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll" event={"ID":"cb970e05-080d-4383-9003-2380f254d3ec","Type":"ContainerDied","Data":"5ca19e1f09b57be5c3d2c7b841235f90d16d02a3f879e2b582d922376760401e"}
Oct 30 06:00:05.399658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.397009    5066 scope.go:122] "RemoveContainer" containerID="5ca19e1f09b57be5c3d2c7b841235f90d16d02a3f879e2b582d922376760401e"
Oct 30 06:00:05.401261 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:05.401200    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll_odh-ai-gateway-infra(cb970e05-080d-4383-9003-2380f254d3ec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll" podUID="cb970e05-080d-4383-9003-2380f254d3ec"
Oct 30 06:00:05.414394 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.414360    5066 generic.go:356] "Generic (PLEG): container finished" podID="a0e477b3-de84-4a4d-bedb-f47ea744632a" containerID="71fbb47282c4829bdbdb54d064f3fedfcebc0c7caac1f68c661db3cfe5bd0dbd" exitCode=60
Oct 30 06:00:05.414818 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.414793    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk" event={"ID":"a0e477b3-de84-4a4d-bedb-f47ea744632a","Type":"ContainerDied","Data":"71fbb47282c4829bdbdb54d064f3fedfcebc0c7caac1f68c661db3cfe5bd0dbd"}
Oct 30 06:00:05.415722 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.415454    5066 scope.go:122] "RemoveContainer" containerID="71fbb47282c4829bdbdb54d064f3fedfcebc0c7caac1f68c661db3cfe5bd0dbd"
Oct 30 06:00:05.416355 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:05.416330    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk_odh-ai-gateway-infra(a0e477b3-de84-4a4d-bedb-f47ea744632a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a"
Oct 30 06:00:05.418172 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.418117    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:05.420512 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.420492    5066 generic.go:356] "Generic (PLEG): container finished" podID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" containerID="495aa3b0943f3c457495fee3a9482f2248a692fe6c008b7c5a1058e8f01c0cf9" exitCode=60
Oct 30 06:00:05.420826 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.420806    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4" event={"ID":"314aebbd-980a-49e5-a6e5-8b807c5eb4b8","Type":"ContainerDied","Data":"495aa3b0943f3c457495fee3a9482f2248a692fe6c008b7c5a1058e8f01c0cf9"}
Oct 30 06:00:05.421442 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.421401    5066 scope.go:122] "RemoveContainer" containerID="495aa3b0943f3c457495fee3a9482f2248a692fe6c008b7c5a1058e8f01c0cf9"
Oct 30 06:00:05.422036 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:05.422012    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4_odh-ai-gateway-infra(314aebbd-980a-49e5-a6e5-8b807c5eb4b8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8"
Oct 30 06:00:05.424945 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.424925    5066 generic.go:356] "Generic (PLEG): container finished" podID="263ee938-5a4a-43a9-bee6-41c00c03eacd" containerID="3548d8ea450b5d066636519740a5477e2e483f2cf65d9186ffcfad03ab6a8b77" exitCode=60
Oct 30 06:00:05.425096 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.425079    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd" event={"ID":"263ee938-5a4a-43a9-bee6-41c00c03eacd","Type":"ContainerDied","Data":"3548d8ea450b5d066636519740a5477e2e483f2cf65d9186ffcfad03ab6a8b77"}
Oct 30 06:00:05.425666 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.425647    5066 scope.go:122] "RemoveContainer" containerID="3548d8ea450b5d066636519740a5477e2e483f2cf65d9186ffcfad03ab6a8b77"
Oct 30 06:00:05.426249 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:05.426209    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd_odh-ai-gateway-infra(263ee938-5a4a-43a9-bee6-41c00c03eacd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd"
Oct 30 06:00:05.467497 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.467471    5066 scope.go:122] "RemoveContainer" containerID="646894c6b04f965dc49d8232d0311a2ff272970bc502ace9dcf44021122580fb"
Oct 30 06:00:05.505727 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.505699    5066 scope.go:122] "RemoveContainer" containerID="241a8209de5a4e6348b542bb59974386e42d6cb889b6cddd661096ee5c07c9e2"
Oct 30 06:00:05.609737 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.609713    5066 scope.go:122] "RemoveContainer" containerID="f313578bfde0307885e9cd905e283cde7f38c7415cdfa52b2e3a25b5062a313e"
Oct 30 06:00:05.721848 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:05.721806    5066 scope.go:122] "RemoveContainer" containerID="730358c033f803558a7cb372ce0c9b3f78e8860d983ef22658f453d1b0c66d5a"
Oct 30 06:00:06.032436 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:06.032391    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 06:00:06.108825 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:06.108721    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 06:00:06.173992 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:06.173879    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:06.673195 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:06.673053    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:06.693904 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:06.693876    5066 scope.go:122] "RemoveContainer" containerID="78e34d28c8e8da09a96c82bb517944df67c12f69990aff043b280f57f8caaa0b"
Oct 30 06:00:06.694607 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:06.694553    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw_odh-ai-gateway-infra(a7b36512-f641-4b2a-9cf9-f8deade880bc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc"
Oct 30 06:00:06.726028 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:06.725997    5066 scope.go:122] "RemoveContainer" containerID="5ca19e1f09b57be5c3d2c7b841235f90d16d02a3f879e2b582d922376760401e"
Oct 30 06:00:06.726683 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:06.726654    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll_odh-ai-gateway-infra(cb970e05-080d-4383-9003-2380f254d3ec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll" podUID="cb970e05-080d-4383-9003-2380f254d3ec"
Oct 30 06:00:06.753702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:06.753673    5066 scope.go:122] "RemoveContainer" containerID="71fbb47282c4829bdbdb54d064f3fedfcebc0c7caac1f68c661db3cfe5bd0dbd"
Oct 30 06:00:06.754182 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:06.754104    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk_odh-ai-gateway-infra(a0e477b3-de84-4a4d-bedb-f47ea744632a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a"
Oct 30 06:00:06.822617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:06.818531    5066 scope.go:122] "RemoveContainer" containerID="495aa3b0943f3c457495fee3a9482f2248a692fe6c008b7c5a1058e8f01c0cf9"
Oct 30 06:00:06.822617 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:06.818997    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4_odh-ai-gateway-infra(314aebbd-980a-49e5-a6e5-8b807c5eb4b8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8"
Oct 30 06:00:06.909617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:06.907820    5066 scope.go:122] "RemoveContainer" containerID="3548d8ea450b5d066636519740a5477e2e483f2cf65d9186ffcfad03ab6a8b77"
Oct 30 06:00:06.909617 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:06.908642    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd_odh-ai-gateway-infra(263ee938-5a4a-43a9-bee6-41c00c03eacd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd"
Oct 30 06:00:07.073673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:07.052334    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": context deadline exceeded"
Oct 30 06:00:07.108617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:07.096816    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 06:00:07.269793 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:07.269758    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 06:00:07.987070 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:07.987028    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:09.788647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:09.788608    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:10.212709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:10.212622    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:10.869647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:10.869567    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:11.371830 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:11.371787    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:11.682628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:11.678099    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:11.861731 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:11.861701    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:12.144514 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:12.144395    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:12.358763 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:12.358727    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:12.540132 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:12.540096    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:13.357271 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:13.357206    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:13.891353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:13.891323    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:14.134536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:14.134501    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:14.327624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:14.327568    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:15.395159 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:15.395131    5066 scope.go:122] "RemoveContainer" containerID="71fbb47282c4829bdbdb54d064f3fedfcebc0c7caac1f68c661db3cfe5bd0dbd"
Oct 30 06:00:15.602375 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:15.602347    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:15.786660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:15.786608    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:16.190614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:16.190502    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk" event={"ID":"a0e477b3-de84-4a4d-bedb-f47ea744632a","Type":"ContainerStarted","Data":"974c1a0465bb3ac955f120bf89e0abc33336158166dc48034cebbe50a9e29131"}
Oct 30 06:00:16.329217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:16.329178    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:16.410612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:16.405703    5066 scope.go:122] "RemoveContainer" containerID="495aa3b0943f3c457495fee3a9482f2248a692fe6c008b7c5a1058e8f01c0cf9"
Oct 30 06:00:16.552395 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:16.552356    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:17.199565 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:17.199536    5066 generic.go:356] "Generic (PLEG): container finished" podID="a0e477b3-de84-4a4d-bedb-f47ea744632a" containerID="974c1a0465bb3ac955f120bf89e0abc33336158166dc48034cebbe50a9e29131" exitCode=60
Oct 30 06:00:17.199782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:17.199622    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk" event={"ID":"a0e477b3-de84-4a4d-bedb-f47ea744632a","Type":"ContainerDied","Data":"974c1a0465bb3ac955f120bf89e0abc33336158166dc48034cebbe50a9e29131"}
Oct 30 06:00:17.199782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:17.199696    5066 scope.go:122] "RemoveContainer" containerID="71fbb47282c4829bdbdb54d064f3fedfcebc0c7caac1f68c661db3cfe5bd0dbd"
Oct 30 06:00:17.200133 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:17.200118    5066 scope.go:122] "RemoveContainer" containerID="974c1a0465bb3ac955f120bf89e0abc33336158166dc48034cebbe50a9e29131"
Oct 30 06:00:17.200539 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:17.200517    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk_odh-ai-gateway-infra(a0e477b3-de84-4a4d-bedb-f47ea744632a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a"
Oct 30 06:00:17.201864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:17.201843    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4" event={"ID":"314aebbd-980a-49e5-a6e5-8b807c5eb4b8","Type":"ContainerStarted","Data":"84862c37a2a8e90afb84be288a6b071ec8b07e80eccac42bc497908b3e1d570c"}
Oct 30 06:00:17.262388 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:17.262358    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"]
Oct 30 06:00:17.395306 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:17.395270    5066 scope.go:122] "RemoveContainer" containerID="78e34d28c8e8da09a96c82bb517944df67c12f69990aff043b280f57f8caaa0b"
Oct 30 06:00:17.395654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:17.395637    5066 scope.go:122] "RemoveContainer" containerID="afae60ac5f523beafa3b357d7c59e0672f0d6baa457224d4bb070886c7d83387"
Oct 30 06:00:17.647537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:17.647510    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:17.832826 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:17.819806    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:18.218732 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.218704    5066 generic.go:356] "Generic (PLEG): container finished" podID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" containerID="84862c37a2a8e90afb84be288a6b071ec8b07e80eccac42bc497908b3e1d570c" exitCode=60
Oct 30 06:00:18.218855 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.218775    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4" event={"ID":"314aebbd-980a-49e5-a6e5-8b807c5eb4b8","Type":"ContainerDied","Data":"84862c37a2a8e90afb84be288a6b071ec8b07e80eccac42bc497908b3e1d570c"}
Oct 30 06:00:18.218855 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.218844    5066 scope.go:122] "RemoveContainer" containerID="495aa3b0943f3c457495fee3a9482f2248a692fe6c008b7c5a1058e8f01c0cf9"
Oct 30 06:00:18.219432 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.219412    5066 scope.go:122] "RemoveContainer" containerID="84862c37a2a8e90afb84be288a6b071ec8b07e80eccac42bc497908b3e1d570c"
Oct 30 06:00:18.219868 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:18.219844    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4_odh-ai-gateway-infra(314aebbd-980a-49e5-a6e5-8b807c5eb4b8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8"
Oct 30 06:00:18.221086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.221064    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2" event={"ID":"926d956d-0651-401b-8661-1cb2bd101f81","Type":"ContainerStarted","Data":"d4b62bc7c7975709351db0783a0e23930da885ef18f96a5a8e1c0ca87054527d"}
Oct 30 06:00:18.222928 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.222906    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw" event={"ID":"a7b36512-f641-4b2a-9cf9-f8deade880bc","Type":"ContainerStarted","Data":"4a105a7bf8a95b163cd08e4e6e99262b378b1c4659344ad65d41f2c84f4e18c9"}
Oct 30 06:00:18.330868 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.330832    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"]
Oct 30 06:00:18.395196 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.395161    5066 scope.go:122] "RemoveContainer" containerID="5ca19e1f09b57be5c3d2c7b841235f90d16d02a3f879e2b582d922376760401e"
Oct 30 06:00:18.395630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.395584    5066 scope.go:122] "RemoveContainer" containerID="3548d8ea450b5d066636519740a5477e2e483f2cf65d9186ffcfad03ab6a8b77"
Oct 30 06:00:18.490385 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.490319    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"
Oct 30 06:00:18.501878 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.501847    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0e477b3-de84-4a4d-bedb-f47ea744632a-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e477b3-de84-4a4d-bedb-f47ea744632a-maas-api-service-ca\") pod \"a0e477b3-de84-4a4d-bedb-f47ea744632a\" (UID: \"a0e477b3-de84-4a4d-bedb-f47ea744632a\") "
Oct 30 06:00:18.502028 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.501987    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0e477b3-de84-4a4d-bedb-f47ea744632a-kube-api-access-xvnv4\" (UniqueName: \"kubernetes.io/projected/a0e477b3-de84-4a4d-bedb-f47ea744632a-kube-api-access-xvnv4\") pod \"a0e477b3-de84-4a4d-bedb-f47ea744632a\" (UID: \"a0e477b3-de84-4a4d-bedb-f47ea744632a\") "
Oct 30 06:00:18.502479 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.502455    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0e477b3-de84-4a4d-bedb-f47ea744632a-maas-api-service-ca" pod "a0e477b3-de84-4a4d-bedb-f47ea744632a" (UID: "a0e477b3-de84-4a4d-bedb-f47ea744632a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:00:18.502875 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.502853    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0e477b3-de84-4a4d-bedb-f47ea744632a-maas-api-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:18.505206 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.505175    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e477b3-de84-4a4d-bedb-f47ea744632a-kube-api-access-xvnv4" pod "a0e477b3-de84-4a4d-bedb-f47ea744632a" (UID: "a0e477b3-de84-4a4d-bedb-f47ea744632a"). InnerVolumeSpecName "kube-api-access-xvnv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:18.604660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.604610    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvnv4\" (UniqueName: \"kubernetes.io/projected/a0e477b3-de84-4a4d-bedb-f47ea744632a-kube-api-access-xvnv4\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:18.781073 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.780893    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:18.983091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:18.983018    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:19.194953 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.194921    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:19.388217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.387874    5066 generic.go:356] "Generic (PLEG): container finished" podID="a7b36512-f641-4b2a-9cf9-f8deade880bc" containerID="4a105a7bf8a95b163cd08e4e6e99262b378b1c4659344ad65d41f2c84f4e18c9" exitCode=60
Oct 30 06:00:19.388217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.387993    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw" event={"ID":"a7b36512-f641-4b2a-9cf9-f8deade880bc","Type":"ContainerDied","Data":"4a105a7bf8a95b163cd08e4e6e99262b378b1c4659344ad65d41f2c84f4e18c9"}
Oct 30 06:00:19.388217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.388087    5066 scope.go:122] "RemoveContainer" containerID="78e34d28c8e8da09a96c82bb517944df67c12f69990aff043b280f57f8caaa0b"
Oct 30 06:00:19.389619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.389042    5066 scope.go:122] "RemoveContainer" containerID="4a105a7bf8a95b163cd08e4e6e99262b378b1c4659344ad65d41f2c84f4e18c9"
Oct 30 06:00:19.395835 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:19.391308    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw_odh-ai-gateway-infra(a7b36512-f641-4b2a-9cf9-f8deade880bc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc"
Oct 30 06:00:19.429626 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.411340    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll" event={"ID":"cb970e05-080d-4383-9003-2380f254d3ec","Type":"ContainerStarted","Data":"d6e274702fad4ab7d86cb63a3a8471dbdc71a61156fd37f40b0d56629acf53b4"}
Oct 30 06:00:19.453708 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.438730    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"]
Oct 30 06:00:19.465658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.462979    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk" event={"ID":"a0e477b3-de84-4a4d-bedb-f47ea744632a","Type":"ContainerDied","Data":"49b7f706727813e0f177d1715dc5f6927790bb595575500b2ce2665777642cfb"}
Oct 30 06:00:19.465658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.463126    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"
Oct 30 06:00:19.496560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.496533    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd" event={"ID":"263ee938-5a4a-43a9-bee6-41c00c03eacd","Type":"ContainerStarted","Data":"3719248257aba8c84d6c06e83a584271c7e0aebcfe6606b462987a7342be1bf1"}
Oct 30 06:00:19.532437 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.532343    5066 generic.go:356] "Generic (PLEG): container finished" podID="926d956d-0651-401b-8661-1cb2bd101f81" containerID="d4b62bc7c7975709351db0783a0e23930da885ef18f96a5a8e1c0ca87054527d" exitCode=60
Oct 30 06:00:19.532437 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.532420    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2" event={"ID":"926d956d-0651-401b-8661-1cb2bd101f81","Type":"ContainerDied","Data":"d4b62bc7c7975709351db0783a0e23930da885ef18f96a5a8e1c0ca87054527d"}
Oct 30 06:00:19.536334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.536166    5066 scope.go:122] "RemoveContainer" containerID="d4b62bc7c7975709351db0783a0e23930da885ef18f96a5a8e1c0ca87054527d"
Oct 30 06:00:19.537546 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:19.536667    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2_odh-ai-gateway-infra(926d956d-0651-401b-8661-1cb2bd101f81)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2" podUID="926d956d-0651-401b-8661-1cb2bd101f81"
Oct 30 06:00:19.571664 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.571551    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"]
Oct 30 06:00:19.578771 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.576908    5066 scope.go:122] "RemoveContainer" containerID="974c1a0465bb3ac955f120bf89e0abc33336158166dc48034cebbe50a9e29131"
Oct 30 06:00:19.608751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.608605    5066 scope.go:122] "RemoveContainer" containerID="afae60ac5f523beafa3b357d7c59e0672f0d6baa457224d4bb070886c7d83387"
Oct 30 06:00:19.668288 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.668253    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"]
Oct 30 06:00:19.677730 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.677696    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-1354a5-29889000-fghsk"]
Oct 30 06:00:19.787243 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.787219    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"
Oct 30 06:00:19.866323 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.866285    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-kube-api-access-wmxdf\" (UniqueName: \"kubernetes.io/projected/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-kube-api-access-wmxdf\") pod \"314aebbd-980a-49e5-a6e5-8b807c5eb4b8\" (UID: \"314aebbd-980a-49e5-a6e5-8b807c5eb4b8\") "
Oct 30 06:00:19.866582 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.866557    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-maas-api-service-ca\") pod \"314aebbd-980a-49e5-a6e5-8b807c5eb4b8\" (UID: \"314aebbd-980a-49e5-a6e5-8b807c5eb4b8\") "
Oct 30 06:00:19.866940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.866912    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-maas-api-service-ca" pod "314aebbd-980a-49e5-a6e5-8b807c5eb4b8" (UID: "314aebbd-980a-49e5-a6e5-8b807c5eb4b8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:00:19.867228 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.867210    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-maas-api-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:19.868926 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.868903    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-kube-api-access-wmxdf" pod "314aebbd-980a-49e5-a6e5-8b807c5eb4b8" (UID: "314aebbd-980a-49e5-a6e5-8b807c5eb4b8"). InnerVolumeSpecName "kube-api-access-wmxdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:19.968539 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:19.968448    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmxdf\" (UniqueName: \"kubernetes.io/projected/314aebbd-980a-49e5-a6e5-8b807c5eb4b8-kube-api-access-wmxdf\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:20.404392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.404360    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a" path="/var/lib/kubelet/pods/a0e477b3-de84-4a4d-bedb-f47ea744632a/volumes"
Oct 30 06:00:20.545464 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.545430    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"
Oct 30 06:00:20.545697 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.545435    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4" event={"ID":"314aebbd-980a-49e5-a6e5-8b807c5eb4b8","Type":"ContainerDied","Data":"6075a4e4e13367a94518202f4a1d4ae49c946bdf33455d475532d85b3a582039"}
Oct 30 06:00:20.545697 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.545577    5066 scope.go:122] "RemoveContainer" containerID="84862c37a2a8e90afb84be288a6b071ec8b07e80eccac42bc497908b3e1d570c"
Oct 30 06:00:20.549042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.549018    5066 generic.go:356] "Generic (PLEG): container finished" podID="263ee938-5a4a-43a9-bee6-41c00c03eacd" containerID="3719248257aba8c84d6c06e83a584271c7e0aebcfe6606b462987a7342be1bf1" exitCode=60
Oct 30 06:00:20.549185 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.549084    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd" event={"ID":"263ee938-5a4a-43a9-bee6-41c00c03eacd","Type":"ContainerDied","Data":"3719248257aba8c84d6c06e83a584271c7e0aebcfe6606b462987a7342be1bf1"}
Oct 30 06:00:20.549531 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.549512    5066 scope.go:122] "RemoveContainer" containerID="3719248257aba8c84d6c06e83a584271c7e0aebcfe6606b462987a7342be1bf1"
Oct 30 06:00:20.549943 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:20.549920    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd_odh-ai-gateway-infra(263ee938-5a4a-43a9-bee6-41c00c03eacd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd"
Oct 30 06:00:20.555188 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.555163    5066 generic.go:356] "Generic (PLEG): container finished" podID="cb970e05-080d-4383-9003-2380f254d3ec" containerID="d6e274702fad4ab7d86cb63a3a8471dbdc71a61156fd37f40b0d56629acf53b4" exitCode=60
Oct 30 06:00:20.555302 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.555207    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll" event={"ID":"cb970e05-080d-4383-9003-2380f254d3ec","Type":"ContainerDied","Data":"d6e274702fad4ab7d86cb63a3a8471dbdc71a61156fd37f40b0d56629acf53b4"}
Oct 30 06:00:20.555722 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.555702    5066 scope.go:122] "RemoveContainer" containerID="d6e274702fad4ab7d86cb63a3a8471dbdc71a61156fd37f40b0d56629acf53b4"
Oct 30 06:00:20.556140 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:20.556106    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll_odh-ai-gateway-infra(cb970e05-080d-4383-9003-2380f254d3ec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll" podUID="cb970e05-080d-4383-9003-2380f254d3ec"
Oct 30 06:00:20.559885 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.559862    5066 scope.go:122] "RemoveContainer" containerID="3548d8ea450b5d066636519740a5477e2e483f2cf65d9186ffcfad03ab6a8b77"
Oct 30 06:00:20.564965 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.564939    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"]
Oct 30 06:00:20.578020 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.577770    5066 scope.go:122] "RemoveContainer" containerID="5ca19e1f09b57be5c3d2c7b841235f90d16d02a3f879e2b582d922376760401e"
Oct 30 06:00:20.734233 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.731041    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"]
Oct 30 06:00:20.744532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.744463    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889000-lrgk4"]
Oct 30 06:00:20.748789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.748766    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"]
Oct 30 06:00:20.768030 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.768002    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"
Oct 30 06:00:20.833338 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.833310    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"
Oct 30 06:00:20.885243 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.885214    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7b36512-f641-4b2a-9cf9-f8deade880bc-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7b36512-f641-4b2a-9cf9-f8deade880bc-maas-api-service-ca\") pod \"a7b36512-f641-4b2a-9cf9-f8deade880bc\" (UID: \"a7b36512-f641-4b2a-9cf9-f8deade880bc\") "
Oct 30 06:00:20.885445 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.885429    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/926d956d-0651-401b-8661-1cb2bd101f81-kube-api-access-zz6x9\" (UniqueName: \"kubernetes.io/projected/926d956d-0651-401b-8661-1cb2bd101f81-kube-api-access-zz6x9\") pod \"926d956d-0651-401b-8661-1cb2bd101f81\" (UID: \"926d956d-0651-401b-8661-1cb2bd101f81\") "
Oct 30 06:00:20.885526 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.885463    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7b36512-f641-4b2a-9cf9-f8deade880bc-kube-api-access-v64cv\" (UniqueName: \"kubernetes.io/projected/a7b36512-f641-4b2a-9cf9-f8deade880bc-kube-api-access-v64cv\") pod \"a7b36512-f641-4b2a-9cf9-f8deade880bc\" (UID: \"a7b36512-f641-4b2a-9cf9-f8deade880bc\") "
Oct 30 06:00:20.885662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.885648    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/926d956d-0651-401b-8661-1cb2bd101f81-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/926d956d-0651-401b-8661-1cb2bd101f81-maas-api-service-ca\") pod \"926d956d-0651-401b-8661-1cb2bd101f81\" (UID: \"926d956d-0651-401b-8661-1cb2bd101f81\") "
Oct 30 06:00:20.885777 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.885748    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b36512-f641-4b2a-9cf9-f8deade880bc-maas-api-service-ca" pod "a7b36512-f641-4b2a-9cf9-f8deade880bc" (UID: "a7b36512-f641-4b2a-9cf9-f8deade880bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:00:20.886251 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.886231    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7b36512-f641-4b2a-9cf9-f8deade880bc-maas-api-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:20.886533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.886514    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/926d956d-0651-401b-8661-1cb2bd101f81-maas-api-service-ca" pod "926d956d-0651-401b-8661-1cb2bd101f81" (UID: "926d956d-0651-401b-8661-1cb2bd101f81"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:00:20.889146 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.889102    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b36512-f641-4b2a-9cf9-f8deade880bc-kube-api-access-v64cv" pod "a7b36512-f641-4b2a-9cf9-f8deade880bc" (UID: "a7b36512-f641-4b2a-9cf9-f8deade880bc"). InnerVolumeSpecName "kube-api-access-v64cv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:20.889244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.889207    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926d956d-0651-401b-8661-1cb2bd101f81-kube-api-access-zz6x9" pod "926d956d-0651-401b-8661-1cb2bd101f81" (UID: "926d956d-0651-401b-8661-1cb2bd101f81"). InnerVolumeSpecName "kube-api-access-zz6x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:20.987658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.987562    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zz6x9\" (UniqueName: \"kubernetes.io/projected/926d956d-0651-401b-8661-1cb2bd101f81-kube-api-access-zz6x9\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:20.987658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.987612    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v64cv\" (UniqueName: \"kubernetes.io/projected/a7b36512-f641-4b2a-9cf9-f8deade880bc-kube-api-access-v64cv\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:20.987658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:20.987623    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/926d956d-0651-401b-8661-1cb2bd101f81-maas-api-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:21.059510 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.059473    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:21.568779 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.568739    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2" event={"ID":"926d956d-0651-401b-8661-1cb2bd101f81","Type":"ContainerDied","Data":"83bf5a64e61640574b5a1edb31717ced718741d67628c64af500882714fe977f"}
Oct 30 06:00:21.568779 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.568767    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"
Oct 30 06:00:21.569062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.568820    5066 scope.go:122] "RemoveContainer" containerID="d4b62bc7c7975709351db0783a0e23930da885ef18f96a5a8e1c0ca87054527d"
Oct 30 06:00:21.577352 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.577316    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"
Oct 30 06:00:21.577506 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.577345    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw" event={"ID":"a7b36512-f641-4b2a-9cf9-f8deade880bc","Type":"ContainerDied","Data":"b19ce4832c11a322d7ee25af3f15c2bb04405b82c2cee3bed0755c0e6687c9f3"}
Oct 30 06:00:21.589748 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.589718    5066 scope.go:122] "RemoveContainer" containerID="4a105a7bf8a95b163cd08e4e6e99262b378b1c4659344ad65d41f2c84f4e18c9"
Oct 30 06:00:21.760411 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.759496    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"]
Oct 30 06:00:21.765792 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.765762    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c78066-29889000-kl6kw"]
Oct 30 06:00:21.814903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.814856    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"]
Oct 30 06:00:21.816250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.815965    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"
Oct 30 06:00:21.820717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.820647    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-4a68a1-29889000-69tq2"]
Oct 30 06:00:21.843982 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.843948    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:21.860577 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.860546    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"
Oct 30 06:00:21.914103 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.914056    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/263ee938-5a4a-43a9-bee6-41c00c03eacd-kube-api-access-565hg\" (UniqueName: \"kubernetes.io/projected/263ee938-5a4a-43a9-bee6-41c00c03eacd-kube-api-access-565hg\") pod \"263ee938-5a4a-43a9-bee6-41c00c03eacd\" (UID: \"263ee938-5a4a-43a9-bee6-41c00c03eacd\") "
Oct 30 06:00:21.914365 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.914341    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb970e05-080d-4383-9003-2380f254d3ec-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb970e05-080d-4383-9003-2380f254d3ec-maas-api-service-ca\") pod \"cb970e05-080d-4383-9003-2380f254d3ec\" (UID: \"cb970e05-080d-4383-9003-2380f254d3ec\") "
Oct 30 06:00:21.914496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.914394    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb970e05-080d-4383-9003-2380f254d3ec-kube-api-access-wvc9z\" (UniqueName: \"kubernetes.io/projected/cb970e05-080d-4383-9003-2380f254d3ec-kube-api-access-wvc9z\") pod \"cb970e05-080d-4383-9003-2380f254d3ec\" (UID: \"cb970e05-080d-4383-9003-2380f254d3ec\") "
Oct 30 06:00:21.914496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.914459    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/263ee938-5a4a-43a9-bee6-41c00c03eacd-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/263ee938-5a4a-43a9-bee6-41c00c03eacd-maas-api-service-ca\") pod \"263ee938-5a4a-43a9-bee6-41c00c03eacd\" (UID: \"263ee938-5a4a-43a9-bee6-41c00c03eacd\") "
Oct 30 06:00:21.915776 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.915749    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb970e05-080d-4383-9003-2380f254d3ec-maas-api-service-ca" pod "cb970e05-080d-4383-9003-2380f254d3ec" (UID: "cb970e05-080d-4383-9003-2380f254d3ec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:00:21.915983 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.915961    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/263ee938-5a4a-43a9-bee6-41c00c03eacd-maas-api-service-ca" pod "263ee938-5a4a-43a9-bee6-41c00c03eacd" (UID: "263ee938-5a4a-43a9-bee6-41c00c03eacd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:00:21.918489 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.918459    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/263ee938-5a4a-43a9-bee6-41c00c03eacd-kube-api-access-565hg" pod "263ee938-5a4a-43a9-bee6-41c00c03eacd" (UID: "263ee938-5a4a-43a9-bee6-41c00c03eacd"). InnerVolumeSpecName "kube-api-access-565hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:21.918786 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:21.918655    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb970e05-080d-4383-9003-2380f254d3ec-kube-api-access-wvc9z" pod "cb970e05-080d-4383-9003-2380f254d3ec" (UID: "cb970e05-080d-4383-9003-2380f254d3ec"). InnerVolumeSpecName "kube-api-access-wvc9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:22.016760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.016727    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-565hg\" (UniqueName: \"kubernetes.io/projected/263ee938-5a4a-43a9-bee6-41c00c03eacd-kube-api-access-565hg\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:22.016760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.016757    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb970e05-080d-4383-9003-2380f254d3ec-maas-api-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:22.016760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.016768    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvc9z\" (UniqueName: \"kubernetes.io/projected/cb970e05-080d-4383-9003-2380f254d3ec-kube-api-access-wvc9z\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:22.017021 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.016777    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/263ee938-5a4a-43a9-bee6-41c00c03eacd-maas-api-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:22.074232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.074090    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:22.403233 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.403209    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" path="/var/lib/kubelet/pods/314aebbd-980a-49e5-a6e5-8b807c5eb4b8/volumes"
Oct 30 06:00:22.403822 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.403804    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="926d956d-0651-401b-8661-1cb2bd101f81" path="/var/lib/kubelet/pods/926d956d-0651-401b-8661-1cb2bd101f81/volumes"
Oct 30 06:00:22.404396 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.404378    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc" path="/var/lib/kubelet/pods/a7b36512-f641-4b2a-9cf9-f8deade880bc/volumes"
Oct 30 06:00:22.593845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.593817    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"
Oct 30 06:00:22.594035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.593812    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd" event={"ID":"263ee938-5a4a-43a9-bee6-41c00c03eacd","Type":"ContainerDied","Data":"4be6a2f05add32c8e9effa33b912b522126f06f4a3dd42ce9e64ada9de5e0e74"}
Oct 30 06:00:22.594035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.593965    5066 scope.go:122] "RemoveContainer" containerID="3719248257aba8c84d6c06e83a584271c7e0aebcfe6606b462987a7342be1bf1"
Oct 30 06:00:22.598632 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.598567    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"
Oct 30 06:00:22.598768 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.598642    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll" event={"ID":"cb970e05-080d-4383-9003-2380f254d3ec","Type":"ContainerDied","Data":"deae1352ce81481c9622e68fb79c5379c9b35b605796f69f722b0c911e03b730"}
Oct 30 06:00:22.605648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.605623    5066 scope.go:122] "RemoveContainer" containerID="d6e274702fad4ab7d86cb63a3a8471dbdc71a61156fd37f40b0d56629acf53b4"
Oct 30 06:00:22.660111 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.660041    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"]
Oct 30 06:00:22.663744 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.663718    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-f22a52-29889000-5rqll"]
Oct 30 06:00:22.718468 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.718077    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"]
Oct 30 06:00:22.721778 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.721752    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889000-2q2nd"]
Oct 30 06:00:22.950339 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.950240    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:22.966737 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.966703    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"]
Oct 30 06:00:22.966998 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:22.966973    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" podUID="414bfcf3-b693-454e-be82-464207f3ede4" containerName="istio-proxy" containerID="cri-o://6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c" gracePeriod=30
Oct 30 06:00:24.403673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:24.403637    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd" path="/var/lib/kubelet/pods/263ee938-5a4a-43a9-bee6-41c00c03eacd/volumes"
Oct 30 06:00:24.404123 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:24.404107    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb970e05-080d-4383-9003-2380f254d3ec" path="/var/lib/kubelet/pods/cb970e05-080d-4383-9003-2380f254d3ec/volumes"
Oct 30 06:00:25.015217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:25.015187    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:26.539057 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:26.539018    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:28.100903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.100866    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:28.314160 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.314135    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 06:00:28.362105 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.362064    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-envoy\") pod \"414bfcf3-b693-454e-be82-464207f3ede4\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") "
Oct 30 06:00:28.362284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.362169    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-kube-api-access-khs6v\" (UniqueName: \"kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-kube-api-access-khs6v\") pod \"414bfcf3-b693-454e-be82-464207f3ede4\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") "
Oct 30 06:00:28.362284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.362222    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-data\") pod \"414bfcf3-b693-454e-be82-464207f3ede4\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") "
Oct 30 06:00:28.362284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.362278    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/414bfcf3-b693-454e-be82-464207f3ede4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414bfcf3-b693-454e-be82-464207f3ede4-istiod-ca-cert\") pod \"414bfcf3-b693-454e-be82-464207f3ede4\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") "
Oct 30 06:00:28.362452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.362412    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-credential-socket\") pod \"414bfcf3-b693-454e-be82-464207f3ede4\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") "
Oct 30 06:00:28.362503 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.362453    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-certs\") pod \"414bfcf3-b693-454e-be82-464207f3ede4\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") "
Oct 30 06:00:28.362503 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.362484    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-istio-token\" (UniqueName: \"kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-istio-token\") pod \"414bfcf3-b693-454e-be82-464207f3ede4\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") "
Oct 30 06:00:28.362653 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.362504    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-socket\") pod \"414bfcf3-b693-454e-be82-464207f3ede4\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") "
Oct 30 06:00:28.362653 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.362625    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/414bfcf3-b693-454e-be82-464207f3ede4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414bfcf3-b693-454e-be82-464207f3ede4-istio-podinfo\") pod \"414bfcf3-b693-454e-be82-464207f3ede4\" (UID: \"414bfcf3-b693-454e-be82-464207f3ede4\") "
Oct 30 06:00:28.363154 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.362991    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-credential-socket" pod "414bfcf3-b693-454e-be82-464207f3ede4" (UID: "414bfcf3-b693-454e-be82-464207f3ede4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:28.363487 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.363318    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-data" pod "414bfcf3-b693-454e-be82-464207f3ede4" (UID: "414bfcf3-b693-454e-be82-464207f3ede4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:28.363576 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.363492    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:28.363576 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.363514    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:28.363576 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.363556    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-socket" pod "414bfcf3-b693-454e-be82-464207f3ede4" (UID: "414bfcf3-b693-454e-be82-464207f3ede4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:28.363941 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.363788    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/414bfcf3-b693-454e-be82-464207f3ede4-istiod-ca-cert" pod "414bfcf3-b693-454e-be82-464207f3ede4" (UID: "414bfcf3-b693-454e-be82-464207f3ede4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:00:28.363941 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.363807    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-certs" pod "414bfcf3-b693-454e-be82-464207f3ede4" (UID: "414bfcf3-b693-454e-be82-464207f3ede4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:28.364943 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.364913    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-envoy" pod "414bfcf3-b693-454e-be82-464207f3ede4" (UID: "414bfcf3-b693-454e-be82-464207f3ede4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:28.365126 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.365096    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-kube-api-access-khs6v" pod "414bfcf3-b693-454e-be82-464207f3ede4" (UID: "414bfcf3-b693-454e-be82-464207f3ede4"). InnerVolumeSpecName "kube-api-access-khs6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:28.366895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.366870    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-istio-token" pod "414bfcf3-b693-454e-be82-464207f3ede4" (UID: "414bfcf3-b693-454e-be82-464207f3ede4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:28.367576 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.367558    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/414bfcf3-b693-454e-be82-464207f3ede4-istio-podinfo" pod "414bfcf3-b693-454e-be82-464207f3ede4" (UID: "414bfcf3-b693-454e-be82-464207f3ede4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:00:28.465165 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.465133    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khs6v\" (UniqueName: \"kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-kube-api-access-khs6v\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:28.465165 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.465163    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/414bfcf3-b693-454e-be82-464207f3ede4-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:28.465165 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.465172    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:28.465400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.465180    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/414bfcf3-b693-454e-be82-464207f3ede4-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:28.465400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.465188    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:28.465400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.465196    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/414bfcf3-b693-454e-be82-464207f3ede4-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:28.465400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.465204    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/414bfcf3-b693-454e-be82-464207f3ede4-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:28.650518 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.650488    5066 generic.go:356] "Generic (PLEG): container finished" podID="414bfcf3-b693-454e-be82-464207f3ede4" containerID="6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c" exitCode=0
Oct 30 06:00:28.650734 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.650532    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"
Oct 30 06:00:28.650734 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.650567    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" event={"ID":"414bfcf3-b693-454e-be82-464207f3ede4","Type":"ContainerDied","Data":"6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c"}
Oct 30 06:00:28.650734 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.650647    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl" event={"ID":"414bfcf3-b693-454e-be82-464207f3ede4","Type":"ContainerDied","Data":"a4f455843b790f0772f3078b694b3a483fca92ffd6d4052729147d5fe364de31"}
Oct 30 06:00:28.650734 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.650672    5066 scope.go:122] "RemoveContainer" containerID="6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c"
Oct 30 06:00:28.661690 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.661671    5066 scope.go:122] "RemoveContainer" containerID="6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c"
Oct 30 06:00:28.661951 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:28.661933    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c\": container with ID starting with 6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c not found: ID does not exist" containerID="6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c"
Oct 30 06:00:28.662014 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.661966    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c"} err="failed to get container status \"6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c\": rpc error: code = NotFound desc = could not find container \"6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c\": container with ID starting with 6b5cb11b39985667361bee09d1f8f4614a402d35f88551220413245a546a249c not found: ID does not exist"
Oct 30 06:00:28.710820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.710790    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"]
Oct 30 06:00:28.724939 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:28.724917    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-ff5kl"]
Oct 30 06:00:29.564177 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:29.564145    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:29.849404 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:29.849330    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:30.016784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:30.016745    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:30.403035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:30.402995    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="414bfcf3-b693-454e-be82-464207f3ede4" path="/var/lib/kubelet/pods/414bfcf3-b693-454e-be82-464207f3ede4/volumes"
Oct 30 06:00:31.545030 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:31.542782    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:32.553090 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:32.553056    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:00:33.132994 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:33.132949    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:34.978828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:34.978782    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:36.561227 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:36.561027    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:36.797772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:36.797731    5066 scope.go:122] "RemoveContainer" containerID="57ec19c9b0e6cb0b7bf5e05def2387ec9606b11474808a89d5b98eef639e838c"
Oct 30 06:00:38.123257 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:38.123223    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:39.931339 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:39.931299    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:41.518036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:41.517920    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:43.151707 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:43.151674    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:44.985659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:44.985536    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:46.606135 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:46.606041    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:46.772094 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:46.772055    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:47.112878 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:47.112844    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:48.591430 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:48.591385    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:48.795329 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:48.795300    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:48.973251 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:48.973214    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:50.798996 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:50.798961    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:52.580505 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:52.580466    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:52.708386 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:52.708353    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"]
Oct 30 06:00:52.708633 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:52.708608    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy" containerID="cri-o://f314207bb87155d59cbdb8e85e49b7f406a5ac9ac3c5d5cc733f526a7c913b89" gracePeriod=30
Oct 30 06:00:52.781990 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:52.781957    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:52.964414 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:52.964383    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:53.113530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:53.113499    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"]
Oct 30 06:00:53.113789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:53.113744    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerName="istio-proxy" containerID="cri-o://12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542" gracePeriod=30
Oct 30 06:00:54.472578 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:54.472539    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:56.027319 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:56.027289    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:56.416529 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:56.416488    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:56.782974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:56.782876    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:57.898807 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.898768    5066 generic.go:356] "Generic (PLEG): container finished" podID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerID="f314207bb87155d59cbdb8e85e49b7f406a5ac9ac3c5d5cc733f526a7c913b89" exitCode=0
Oct 30 06:00:57.899312 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.898842    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" event={"ID":"366b7e0f-6d47-4b8e-aeb1-98503d89807a","Type":"ContainerDied","Data":"f314207bb87155d59cbdb8e85e49b7f406a5ac9ac3c5d5cc733f526a7c913b89"}
Oct 30 06:00:57.959618 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.959573    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 06:00:57.961708 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.961679    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-credential-socket\") pod \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") "
Oct 30 06:00:57.961867 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.961767    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-socket\") pod \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") "
Oct 30 06:00:57.961940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.961924    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-certs\") pod \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") "
Oct 30 06:00:57.961986 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.961969    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-envoy\") pod \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") "
Oct 30 06:00:57.962044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.961995    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-credential-socket" pod "366b7e0f-6d47-4b8e-aeb1-98503d89807a" (UID: "366b7e0f-6d47-4b8e-aeb1-98503d89807a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:57.962044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.962017    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-data\") pod \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") "
Oct 30 06:00:57.962159 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.962059    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-podinfo\") pod \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") "
Oct 30 06:00:57.962159 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.962110    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-token\" (UniqueName: \"kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-token\") pod \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") "
Oct 30 06:00:57.962159 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.962096    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-socket" pod "366b7e0f-6d47-4b8e-aeb1-98503d89807a" (UID: "366b7e0f-6d47-4b8e-aeb1-98503d89807a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:57.962366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.962211    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-kube-api-access-8vr5p\" (UniqueName: \"kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-kube-api-access-8vr5p\") pod \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") "
Oct 30 06:00:57.962366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.962248    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istiod-ca-cert\") pod \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\" (UID: \"366b7e0f-6d47-4b8e-aeb1-98503d89807a\") "
Oct 30 06:00:57.962489 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.962443    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-certs" pod "366b7e0f-6d47-4b8e-aeb1-98503d89807a" (UID: "366b7e0f-6d47-4b8e-aeb1-98503d89807a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:57.963145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.963085    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:57.963145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.963111    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:57.963145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.963127    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:57.963145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.962969    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istiod-ca-cert" pod "366b7e0f-6d47-4b8e-aeb1-98503d89807a" (UID: "366b7e0f-6d47-4b8e-aeb1-98503d89807a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:00:57.964664 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.964582    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-podinfo" pod "366b7e0f-6d47-4b8e-aeb1-98503d89807a" (UID: "366b7e0f-6d47-4b8e-aeb1-98503d89807a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:00:57.964861 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.964839    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-kube-api-access-8vr5p" pod "366b7e0f-6d47-4b8e-aeb1-98503d89807a" (UID: "366b7e0f-6d47-4b8e-aeb1-98503d89807a"). InnerVolumeSpecName "kube-api-access-8vr5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:57.964944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.964930    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-token" pod "366b7e0f-6d47-4b8e-aeb1-98503d89807a" (UID: "366b7e0f-6d47-4b8e-aeb1-98503d89807a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:57.965702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.965668    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-data" pod "366b7e0f-6d47-4b8e-aeb1-98503d89807a" (UID: "366b7e0f-6d47-4b8e-aeb1-98503d89807a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:57.965702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:57.965690    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-envoy" pod "366b7e0f-6d47-4b8e-aeb1-98503d89807a" (UID: "366b7e0f-6d47-4b8e-aeb1-98503d89807a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:58.065183 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.065085    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.065183 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.065133    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.065183 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.065148    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.065183 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.065162    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.065183 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.065175    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vr5p\" (UniqueName: \"kubernetes.io/projected/366b7e0f-6d47-4b8e-aeb1-98503d89807a-kube-api-access-8vr5p\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.065477 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.065189    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/366b7e0f-6d47-4b8e-aeb1-98503d89807a-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.414305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.411000    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:58.427611 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.427230    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 06:00:58.471721 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.471688    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-podinfo\") pod \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") "
Oct 30 06:00:58.471869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.471737    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-envoy\") pod \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") "
Oct 30 06:00:58.471929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.471891    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-kube-api-access-snflv\" (UniqueName: \"kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-kube-api-access-snflv\") pod \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") "
Oct 30 06:00:58.471974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.471950    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-data\") pod \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") "
Oct 30 06:00:58.472039 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.472019    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-credential-socket\") pod \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") "
Oct 30 06:00:58.472619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.472152    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-socket\") pod \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") "
Oct 30 06:00:58.472619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.472265    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-credential-socket" pod "39979b3b-2512-4d2f-b30f-6f59b843e2d6" (UID: "39979b3b-2512-4d2f-b30f-6f59b843e2d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:58.472619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.472276    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-data" pod "39979b3b-2512-4d2f-b30f-6f59b843e2d6" (UID: "39979b3b-2512-4d2f-b30f-6f59b843e2d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:58.472619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.472274    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istiod-ca-cert\") pod \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") "
Oct 30 06:00:58.472619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.472422    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-token\" (UniqueName: \"kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-token\") pod \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") "
Oct 30 06:00:58.472619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.472472    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-certs\") pod \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\" (UID: \"39979b3b-2512-4d2f-b30f-6f59b843e2d6\") "
Oct 30 06:00:58.472619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.472391    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-socket" pod "39979b3b-2512-4d2f-b30f-6f59b843e2d6" (UID: "39979b3b-2512-4d2f-b30f-6f59b843e2d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:58.472619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.472579    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istiod-ca-cert" pod "39979b3b-2512-4d2f-b30f-6f59b843e2d6" (UID: "39979b3b-2512-4d2f-b30f-6f59b843e2d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:00:58.473160 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.473099    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-certs" pod "39979b3b-2512-4d2f-b30f-6f59b843e2d6" (UID: "39979b3b-2512-4d2f-b30f-6f59b843e2d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:58.473925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.473640    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.473925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.473665    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.473925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.473680    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.473925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.473695    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.473925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.473710    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.489003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.488961    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-podinfo" pod "39979b3b-2512-4d2f-b30f-6f59b843e2d6" (UID: "39979b3b-2512-4d2f-b30f-6f59b843e2d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:00:58.499024 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.498992    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-envoy" pod "39979b3b-2512-4d2f-b30f-6f59b843e2d6" (UID: "39979b3b-2512-4d2f-b30f-6f59b843e2d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:00:58.499024 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.499017    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-token" pod "39979b3b-2512-4d2f-b30f-6f59b843e2d6" (UID: "39979b3b-2512-4d2f-b30f-6f59b843e2d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:58.499207 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.499046    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-kube-api-access-snflv" pod "39979b3b-2512-4d2f-b30f-6f59b843e2d6" (UID: "39979b3b-2512-4d2f-b30f-6f59b843e2d6"). InnerVolumeSpecName "kube-api-access-snflv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:00:58.575440 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.575401    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.575440 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.575432    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.575440 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.575440    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39979b3b-2512-4d2f-b30f-6f59b843e2d6-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.575797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.575451    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snflv\" (UniqueName: \"kubernetes.io/projected/39979b3b-2512-4d2f-b30f-6f59b843e2d6-kube-api-access-snflv\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:00:58.907971 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.907923    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"
Oct 30 06:00:58.908483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.907928    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl" event={"ID":"366b7e0f-6d47-4b8e-aeb1-98503d89807a","Type":"ContainerDied","Data":"c4a681a630927bded30181f61f57b0c06749bb510cc3d4dd9dbde355fad25f0a"}
Oct 30 06:00:58.908483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.908136    5066 scope.go:122] "RemoveContainer" containerID="f314207bb87155d59cbdb8e85e49b7f406a5ac9ac3c5d5cc733f526a7c913b89"
Oct 30 06:00:58.909703 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.909528    5066 generic.go:356] "Generic (PLEG): container finished" podID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerID="12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542" exitCode=0
Oct 30 06:00:58.909703 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.909577    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"
Oct 30 06:00:58.909703 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.909633    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" event={"ID":"39979b3b-2512-4d2f-b30f-6f59b843e2d6","Type":"ContainerDied","Data":"12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542"}
Oct 30 06:00:58.909703 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.909675    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg" event={"ID":"39979b3b-2512-4d2f-b30f-6f59b843e2d6","Type":"ContainerDied","Data":"f01765f15736faa66f14f43f90e26c736c8a6ab181d8825ccecf4b69900f395f"}
Oct 30 06:00:58.920061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.920014    5066 scope.go:122] "RemoveContainer" containerID="12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542"
Oct 30 06:00:58.930199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.930155    5066 scope.go:122] "RemoveContainer" containerID="12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542"
Oct 30 06:00:58.930500 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:00:58.930474    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542\": container with ID starting with 12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542 not found: ID does not exist" containerID="12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542"
Oct 30 06:00:58.930578 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.930508    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542"} err="failed to get container status \"12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542\": rpc error: code = NotFound desc = could not find container \"12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542\": container with ID starting with 12caad79324bed3a33e3b41a4c2ca486b9ab29dbb52d592e66425332eb4b1542 not found: ID does not exist"
Oct 30 06:00:58.971696 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.971668    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"]
Oct 30 06:00:58.987209 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:58.987169    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-w4wbg"]
Oct 30 06:00:59.050733 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:59.050650    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"]
Oct 30 06:00:59.062014 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:59.061978    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-tg9fl"]
Oct 30 06:00:59.518337 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:59.518298    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:00:59.857164 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:00:59.857065    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:00.403148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:00.403111    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" path="/var/lib/kubelet/pods/366b7e0f-6d47-4b8e-aeb1-98503d89807a/volumes"
Oct 30 06:01:00.403713 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:00.403687    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" path="/var/lib/kubelet/pods/39979b3b-2512-4d2f-b30f-6f59b843e2d6/volumes"
Oct 30 06:01:00.549366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:00.547373    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:02.332232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:02.332186    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:02.555949 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:01:02.555917    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:01:02.969702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:02.969672    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:03.633499 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:03.633467    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:03.882202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:03.882167    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:04.318471 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:04.318440    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:06.455216 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:06.455185    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:08.323758 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:08.323720    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:08.546140 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:08.546106    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:08.845759 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:08.845728    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:10.220694 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:10.220656    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:10.432760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:10.432686    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:11.131635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:11.131580    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:13.100537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:13.100501    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:14.913451 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:14.913421    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:15.670378 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:15.670328    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:16.212949 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:16.212885    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:16.758674 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:16.758619    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:16.972146 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:16.972110    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:18.898037 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:18.897998    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:20.531252 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:20.531139    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:20.821680 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:20.821576    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:21.032328 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:21.032291    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:22.302865 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:22.302819    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:22.762613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:22.737467    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:23.143538 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:23.143440    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:25.345289 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:25.345248    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:27.144287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:27.144254    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:28.889575 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:28.889494    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:29.771502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:29.771468    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:30.729450 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:30.729420    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:32.530788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:32.530736    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:32.561572 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:01:32.561537    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:01:34.475856 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:34.475799    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:36.392718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:36.392674    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:36.688217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:36.688189    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:01:36.693709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:36.693678    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:01:37.010019 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:37.009903    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:37.295512 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:37.295089    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:37.480633 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:37.480482    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:38.143289 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:38.143253    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:40.096899 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:40.096865    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:42.213788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:42.213756    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:42.966911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:42.966872    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:44.133561 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.133524    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"]
Oct 30 06:01:44.134217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134200    5066 cpu_manager.go:414] "containerMap: removing container" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd" containerName="cleanup"
Oct 30 06:01:44.134217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134218    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134236    5066 cpu_manager.go:414] "containerMap: removing container" podUID="926d956d-0651-401b-8661-1cb2bd101f81" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134241    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="926d956d-0651-401b-8661-1cb2bd101f81" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134251    5066 cpu_manager.go:414] "containerMap: removing container" podUID="cb970e05-080d-4383-9003-2380f254d3ec" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134256    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb970e05-080d-4383-9003-2380f254d3ec" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134265    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134270    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134281    5066 cpu_manager.go:414] "containerMap: removing container" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134285    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134296    5066 cpu_manager.go:414] "containerMap: removing container" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134301    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134309    5066 cpu_manager.go:414] "containerMap: removing container" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134314    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134329    5066 cpu_manager.go:414] "containerMap: removing container" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134333    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134343    5066 cpu_manager.go:414] "containerMap: removing container" podUID="926d956d-0651-401b-8661-1cb2bd101f81" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134347    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="926d956d-0651-401b-8661-1cb2bd101f81" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134354    5066 cpu_manager.go:414] "containerMap: removing container" podUID="414bfcf3-b693-454e-be82-464207f3ede4" containerName="istio-proxy"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134358    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="414bfcf3-b693-454e-be82-464207f3ede4" containerName="istio-proxy"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134365    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc" containerName="cleanup"
Oct 30 06:01:44.134356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134369    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134380    5066 cpu_manager.go:414] "containerMap: removing container" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134385    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134395    5066 cpu_manager.go:414] "containerMap: removing container" podUID="ea0227f2-0020-4aa2-b62b-843fcfbeafe1" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134401    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea0227f2-0020-4aa2-b62b-843fcfbeafe1" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134410    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134417    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134426    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134430    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134435    5066 cpu_manager.go:414] "containerMap: removing container" podUID="926d956d-0651-401b-8661-1cb2bd101f81" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134439    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="926d956d-0651-401b-8661-1cb2bd101f81" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134452    5066 cpu_manager.go:414] "containerMap: removing container" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134456    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134465    5066 cpu_manager.go:414] "containerMap: removing container" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerName="istio-proxy"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134469    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerName="istio-proxy"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134482    5066 cpu_manager.go:414] "containerMap: removing container" podUID="cb970e05-080d-4383-9003-2380f254d3ec" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134486    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb970e05-080d-4383-9003-2380f254d3ec" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134618    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="39979b3b-2512-4d2f-b30f-6f59b843e2d6" containerName="istio-proxy"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134630    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134637    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb970e05-080d-4383-9003-2380f254d3ec" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134644    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134651    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="926d956d-0651-401b-8661-1cb2bd101f81" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134656    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb970e05-080d-4383-9003-2380f254d3ec" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134662    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134668    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134674    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134680    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134686    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="414bfcf3-b693-454e-be82-464207f3ede4" containerName="istio-proxy"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134701    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134708    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="366b7e0f-6d47-4b8e-aeb1-98503d89807a" containerName="istio-proxy"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134716    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea0227f2-0020-4aa2-b62b-843fcfbeafe1" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134722    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134726    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="926d956d-0651-401b-8661-1cb2bd101f81" containerName="cleanup"
Oct 30 06:01:44.135360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.134732    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc" containerName="cleanup"
Oct 30 06:01:44.139860 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.139828    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.167446 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.167412    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-387b16-openshift-default-dockercfg-hxplc\""
Oct 30 06:01:44.180898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.180865    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"]
Oct 30 06:01:44.220366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.220333    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-socket\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.220540 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.220474    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qckmf\" (UniqueName: \"kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-kube-api-access-qckmf\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.220640 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.220546    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-certs\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.220704 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.220665    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-podinfo\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.220704 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.220693    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51408510-bf0c-4ad2-b776-6dff1feb777b-istiod-ca-cert\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.220811 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.220796    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-token\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.220858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.220826    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-envoy\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.220995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.220974    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-credential-socket\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.221174 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.221003    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-data\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.322035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322000    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qckmf\" (UniqueName: \"kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-kube-api-access-qckmf\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.322214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322071    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-certs\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.322214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322091    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-podinfo\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.322214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322114    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51408510-bf0c-4ad2-b776-6dff1feb777b-istiod-ca-cert\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.322214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322178    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-token\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.322214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322202    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-envoy\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.322486 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322350    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-credential-socket\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.322486 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322374    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-data\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.322486 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322449    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-socket\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.322748 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322721    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-certs\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.322824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322782    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-data\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.323025 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322996    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-credential-socket\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.323025 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.323013    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51408510-bf0c-4ad2-b776-6dff1feb777b-istiod-ca-cert\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.323025 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.322999    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-socket\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.324975 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.324950    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-envoy\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.325100 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.325083    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-podinfo\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.362135 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.362107    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qckmf\" (UniqueName: \"kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-kube-api-access-qckmf\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.362678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.362658    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-token\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.375191 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.375158    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"]
Oct 30 06:01:44.376211 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.376192    5066 cpu_manager.go:414] "containerMap: removing container" podUID="cb970e05-080d-4383-9003-2380f254d3ec" containerName="cleanup"
Oct 30 06:01:44.376276 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.376214    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb970e05-080d-4383-9003-2380f254d3ec" containerName="cleanup"
Oct 30 06:01:44.376276 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.376256    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc" containerName="cleanup"
Oct 30 06:01:44.376276 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.376264    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7b36512-f641-4b2a-9cf9-f8deade880bc" containerName="cleanup"
Oct 30 06:01:44.376380 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.376287    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a" containerName="cleanup"
Oct 30 06:01:44.376380 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.376295    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a" containerName="cleanup"
Oct 30 06:01:44.376485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.376474    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb970e05-080d-4383-9003-2380f254d3ec" containerName="cleanup"
Oct 30 06:01:44.376520 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.376494    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="314aebbd-980a-49e5-a6e5-8b807c5eb4b8" containerName="cleanup"
Oct 30 06:01:44.376520 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.376505    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="263ee938-5a4a-43a9-bee6-41c00c03eacd" containerName="cleanup"
Oct 30 06:01:44.376520 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.376517    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0e477b3-de84-4a4d-bedb-f47ea744632a" containerName="cleanup"
Oct 30 06:01:44.376634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.376527    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="926d956d-0651-401b-8661-1cb2bd101f81" containerName="cleanup"
Oct 30 06:01:44.382030 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.382009    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:44.409306 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.409265    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"]
Oct 30 06:01:44.459065 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.459021    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:44.521820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.521759    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:44.522096 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.522062    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 06:01:44.522096 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.522188    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:01:44.526385 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.526282    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvfck\" (UniqueName: \"kubernetes.io/projected/b54e67a7-ee65-42d7-8496-ca62d710a0e1-kube-api-access-vvfck\") pod \"maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp\" (UID: \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:44.526718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.526689    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp\" (UID: \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:44.526880 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.526827    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp\" (UID: \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:44.632694 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.632351    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp\" (UID: \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:44.632694 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.632448    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp\" (UID: \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:44.632694 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.632666    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvfck\" (UniqueName: \"kubernetes.io/projected/b54e67a7-ee65-42d7-8496-ca62d710a0e1-kube-api-access-vvfck\") pod \"maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp\" (UID: \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:44.633541 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.633231    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp\" (UID: \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:44.636158 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.636137    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp\" (UID: \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:44.671408 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.671365    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvfck\" (UniqueName: \"kubernetes.io/projected/b54e67a7-ee65-42d7-8496-ca62d710a0e1-kube-api-access-vvfck\") pod \"maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp\" (UID: \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:44.714977 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.714943    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:44.859114 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.859086    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"]
Oct 30 06:01:44.860069 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:01:44.860026    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51408510_bf0c_4ad2_b776_6dff1feb777b.slice/crio-7052f3101cfe619813aeb7a7f08abb06cae2e6c50631f763f0143846d98d556e WatchSource:0}: Error finding container 7052f3101cfe619813aeb7a7f08abb06cae2e6c50631f763f0143846d98d556e: Status 404 returned error can't find the container with id 7052f3101cfe619813aeb7a7f08abb06cae2e6c50631f763f0143846d98d556e
Oct 30 06:01:44.863178 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.863143    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:01:44.863251 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.863221    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:01:44.863299 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:44.863252    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:01:45.037017 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:45.036988    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"]
Oct 30 06:01:45.209504 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:45.209422    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:45.305202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:45.305165    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp" event={"ID":"b54e67a7-ee65-42d7-8496-ca62d710a0e1","Type":"ContainerStarted","Data":"7bfa82c74b6f909f025f1a01a79a709ce21dba5618feb488d7d663058bfd2ed8"}
Oct 30 06:01:45.309212 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:45.307659    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" event={"ID":"51408510-bf0c-4ad2-b776-6dff1feb777b","Type":"ContainerStarted","Data":"82c20f1f84ea1c980c48c2925daf1c7fc369b12624f7c357d0eedf1e58e17f98"}
Oct 30 06:01:45.309212 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:45.307718    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" event={"ID":"51408510-bf0c-4ad2-b776-6dff1feb777b","Type":"ContainerStarted","Data":"7052f3101cfe619813aeb7a7f08abb06cae2e6c50631f763f0143846d98d556e"}
Oct 30 06:01:45.380625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:45.372416    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" podStartSLOduration=1.372394333 podStartE2EDuration="1.372394333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:01:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:01:45.371293743 +0000 UTC m=+1809.116579075" watchObservedRunningTime="2026-10-30 06:01:45.372394333 +0000 UTC m=+1809.117679667"
Oct 30 06:01:45.460015 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:45.459926    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:45.462411 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:45.462382    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Oct 30 06:01:45.462671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:45.462646    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Oct 30 06:01:45.498017 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:45.497982    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:45.709254 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:45.709222    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:45.898671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:45.898580    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:46.643935 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:46.643852    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Oct 30 06:01:46.644510 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:46.644479    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Oct 30 06:01:46.644767 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:46.644740    5066 generic.go:356] "Generic (PLEG): container finished" podID="b54e67a7-ee65-42d7-8496-ca62d710a0e1" containerID="7cf908443bfe18bbd45724853e981f5d3c0be34aece98798b8e03b1242e93054" exitCode=0
Oct 30 06:01:46.658616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:46.646422    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:46.789608 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:46.788362    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp" event={"ID":"b54e67a7-ee65-42d7-8496-ca62d710a0e1","Type":"ContainerDied","Data":"7cf908443bfe18bbd45724853e981f5d3c0be34aece98798b8e03b1242e93054"}
Oct 30 06:01:47.144612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.144426    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:47.317887 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.317847    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"]
Oct 30 06:01:47.322371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.322346    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.366461 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.366428    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-de7242-openshift-default-dockercfg-x5rjm\""
Oct 30 06:01:47.380917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.380890    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"]
Oct 30 06:01:47.405725 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.405697    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-socket\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.405928 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.405758    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istiod-ca-cert\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.405928 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.405822    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-certs\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.406051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.405968    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-credential-socket\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.406051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.406010    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-podinfo\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.406126 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.406102    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9qcd\" (UniqueName: \"kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-kube-api-access-x9qcd\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.406230 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.406216    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-token\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.406412 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.406387    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-data\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.406547 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.406445    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-envoy\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.417342 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.417310    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:47.470679 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.469935    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Oct 30 06:01:47.470679 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.470003    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Oct 30 06:01:47.507393 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.507364    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-socket\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.507655 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.507635    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istiod-ca-cert\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.507774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.507753    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-socket\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.507916 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.507896    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-certs\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.508116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.508099    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-credential-socket\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.508292 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.508265    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-credential-socket\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.508292 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.508263    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-podinfo\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.508428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.508339    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9qcd\" (UniqueName: \"kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-kube-api-access-x9qcd\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.508428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.508388    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istiod-ca-cert\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.508428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.508410    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-token\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.508580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.508485    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-data\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.508580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.508523    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-envoy\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.509080 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.508120    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-certs\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.509080 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.509009    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-data\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.510969 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.510949    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-envoy\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.511977 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.511957    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-podinfo\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.559808 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.559743    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-token\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.561442 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.561408    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9qcd\" (UniqueName: \"kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-kube-api-access-x9qcd\") pod \"e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") " pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:47.623250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.623219    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:47.690833 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:47.690802    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:48.305419 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.305390    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:48.369388 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.369349    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b54e67a7-ee65-42d7-8496-ca62d710a0e1-kube-api-access-vvfck\" (UniqueName: \"kubernetes.io/projected/b54e67a7-ee65-42d7-8496-ca62d710a0e1-kube-api-access-vvfck\") pod \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\" (UID: \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\") "
Oct 30 06:01:48.369616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.369491    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-xks-ca\") pod \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\" (UID: \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\") "
Oct 30 06:01:48.369616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.369574    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-openshift-ca\") pod \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\" (UID: \"b54e67a7-ee65-42d7-8496-ca62d710a0e1\") "
Oct 30 06:01:48.370876 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.370843    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-openshift-ca" pod "b54e67a7-ee65-42d7-8496-ca62d710a0e1" (UID: "b54e67a7-ee65-42d7-8496-ca62d710a0e1"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:01:48.373400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.373370    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b54e67a7-ee65-42d7-8496-ca62d710a0e1-kube-api-access-vvfck" pod "b54e67a7-ee65-42d7-8496-ca62d710a0e1" (UID: "b54e67a7-ee65-42d7-8496-ca62d710a0e1"). InnerVolumeSpecName "kube-api-access-vvfck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:01:48.373929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.373908    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-xks-ca" pod "b54e67a7-ee65-42d7-8496-ca62d710a0e1" (UID: "b54e67a7-ee65-42d7-8496-ca62d710a0e1"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:01:48.462650 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.461656    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Oct 30 06:01:48.462650 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.461727    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Oct 30 06:01:48.471645 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.471617    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:01:48.471645 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.471641    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b54e67a7-ee65-42d7-8496-ca62d710a0e1-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:01:48.471900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.471653    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvfck\" (UniqueName: \"kubernetes.io/projected/b54e67a7-ee65-42d7-8496-ca62d710a0e1-kube-api-access-vvfck\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:01:48.533671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.531318    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"]
Oct 30 06:01:48.616094 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:01:48.611332    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb01d6455_9151_4e8f_91e9_ccaa4fe18471.slice/crio-d21d9489d29ff55b5d3ee0fd495382e6282340ebff8aab42d55444d04c97e1f8 WatchSource:0}: Error finding container d21d9489d29ff55b5d3ee0fd495382e6282340ebff8aab42d55444d04c97e1f8: Status 404 returned error can't find the container with id d21d9489d29ff55b5d3ee0fd495382e6282340ebff8aab42d55444d04c97e1f8
Oct 30 06:01:48.650623 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.640660    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:01:48.651020 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.650965    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:01:48.673616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.664723    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:01:48.715505 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.715446    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" event={"ID":"b01d6455-9151-4e8f-91e9-ccaa4fe18471","Type":"ContainerStarted","Data":"d21d9489d29ff55b5d3ee0fd495382e6282340ebff8aab42d55444d04c97e1f8"}
Oct 30 06:01:48.718583 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.717850    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp" event={"ID":"b54e67a7-ee65-42d7-8496-ca62d710a0e1","Type":"ContainerDied","Data":"7bfa82c74b6f909f025f1a01a79a709ce21dba5618feb488d7d663058bfd2ed8"}
Oct 30 06:01:48.718583 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.717884    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7bfa82c74b6f909f025f1a01a79a709ce21dba5618feb488d7d663058bfd2ed8"
Oct 30 06:01:48.718583 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:48.717935    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"
Oct 30 06:01:49.462011 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:49.461964    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Oct 30 06:01:49.462220 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:49.462021    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Oct 30 06:01:49.790739 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:49.790638    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" event={"ID":"b01d6455-9151-4e8f-91e9-ccaa4fe18471","Type":"ContainerStarted","Data":"b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407"}
Oct 30 06:01:49.904059 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:49.904022    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:50.135021 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.128360    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:50.295630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.283749    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" podStartSLOduration=3.283727852 podStartE2EDuration="3.283727852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:01:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:01:49.882388731 +0000 UTC m=+1813.627674066" watchObservedRunningTime="2026-10-30 06:01:50.283727852 +0000 UTC m=+1814.029013184"
Oct 30 06:01:50.510948 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.509937    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Oct 30 06:01:50.510948 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.510012    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Oct 30 06:01:50.519232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.519198    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"]
Oct 30 06:01:50.520729 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.520702    5066 cpu_manager.go:414] "containerMap: removing container" podUID="b54e67a7-ee65-42d7-8496-ca62d710a0e1" containerName="revoke-keys"
Oct 30 06:01:50.520729 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.520727    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b54e67a7-ee65-42d7-8496-ca62d710a0e1" containerName="revoke-keys"
Oct 30 06:01:50.521054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.521040    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="b54e67a7-ee65-42d7-8496-ca62d710a0e1" containerName="revoke-keys"
Oct 30 06:01:50.567908 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.566991    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"]
Oct 30 06:01:50.567908 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.567115    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.671615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.670840    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c1d09f-openshift-default-dockercfg-gpb22\""
Oct 30 06:01:50.691055 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.691020    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:50.692173 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.692123    5066 patch_prober.go:29] interesting pod/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Oct 30 06:01:50.692260 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.692178    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" podUID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Oct 30 06:01:50.720673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.720639    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-token\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.720927 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.720910    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnrxm\" (UniqueName: \"kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-kube-api-access-rnrxm\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.721094 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.721083    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-podinfo\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.721275 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.721262    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-credential-socket\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.721399 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.721388    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-socket\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.721514 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.721484    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-certs\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.721659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.721647    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-data\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.721843 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.721830    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-envoy\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.721965 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.721953    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istiod-ca-cert\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.860674 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.845510    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-envoy\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.861429 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.861397    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istiod-ca-cert\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.862411 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.862387    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istiod-ca-cert\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.862742 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.862721    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-token\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.862978 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.862960    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnrxm\" (UniqueName: \"kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-kube-api-access-rnrxm\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.863179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.863164    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-podinfo\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.863371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.863352    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-credential-socket\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.863543 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.863526    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-socket\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.863684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.863669    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-certs\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.863821 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.863807    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-data\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.864333 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.864313    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-data\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.866030 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.865989    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-credential-socket\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.866956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.866934    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-socket\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.867313 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.867291    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-certs\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.913202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.912480    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-podinfo\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.913202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.912970    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-envoy\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.918151 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.918124    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnrxm\" (UniqueName: \"kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-kube-api-access-rnrxm\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:50.920270 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:50.920243    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-token\") pod \"e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") " pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:51.191231 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.189967    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:51.261237 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.261156    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:51.486056 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.463807    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body=
Oct 30 06:01:51.486056 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.463869    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused"
Oct 30 06:01:51.486056 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.480118    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:51.598743 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:01:51.598714    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab2742b_6ca0_4d4e_981c_6525b3e7b26d.slice/crio-4d3ee8c0881d518745d09814571bdabb180142a2e1a9c83463dca4ec31fde244 WatchSource:0}: Error finding container 4d3ee8c0881d518745d09814571bdabb180142a2e1a9c83463dca4ec31fde244: Status 404 returned error can't find the container with id 4d3ee8c0881d518745d09814571bdabb180142a2e1a9c83463dca4ec31fde244
Oct 30 06:01:51.598879 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.598853    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"]
Oct 30 06:01:51.601643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.601613    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:01:51.601736 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.601688    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:01:51.601736 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.601718    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:01:51.692076 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.692043    5066 patch_prober.go:29] interesting pod/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Oct 30 06:01:51.692372 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.692341    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" podUID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Oct 30 06:01:51.833665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.824573    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" event={"ID":"aab2742b-6ca0-4d4e-981c-6525b3e7b26d","Type":"ContainerStarted","Data":"62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462"}
Oct 30 06:01:51.833665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.824653    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" event={"ID":"aab2742b-6ca0-4d4e-981c-6525b3e7b26d","Type":"ContainerStarted","Data":"4d3ee8c0881d518745d09814571bdabb180142a2e1a9c83463dca4ec31fde244"}
Oct 30 06:01:51.897331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:51.897262    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" podStartSLOduration=1.897240677 podStartE2EDuration="1.897240677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:01:51.89544036 +0000 UTC m=+1815.640725693" watchObservedRunningTime="2026-10-30 06:01:51.897240677 +0000 UTC m=+1815.642526013"
Oct 30 06:01:52.191228 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:52.191192    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:52.193442 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:52.193417    5066 patch_prober.go:29] interesting pod/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Oct 30 06:01:52.193559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:52.193477    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" podUID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Oct 30 06:01:52.541607 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:52.541452    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:52.640615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:52.629881    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:52.644282 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:52.644087    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:52.668024 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:52.667951    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:01:52.692813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:52.692181    5066 patch_prober.go:29] interesting pod/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Oct 30 06:01:52.692813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:52.692253    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" podUID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Oct 30 06:01:53.190906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:53.190864    5066 patch_prober.go:29] interesting pod/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Oct 30 06:01:53.191390 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:53.190941    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" podUID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Oct 30 06:01:53.414872 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:53.414836    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:53.662624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:53.662177    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:53.693614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:53.692071    5066 patch_prober.go:29] interesting pod/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Oct 30 06:01:53.693614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:53.692137    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" podUID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Oct 30 06:01:54.194614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:54.194530    5066 patch_prober.go:29] interesting pod/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Oct 30 06:01:54.194614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:54.194604    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" podUID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Oct 30 06:01:54.691354 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:54.691311    5066 patch_prober.go:29] interesting pod/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Oct 30 06:01:54.691538 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:54.691389    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" podUID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Oct 30 06:01:54.913663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:54.913629    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"]
Oct 30 06:01:54.918429 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:54.918404    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:54.978741 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:54.976914    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-7df9fa-serving-cert\""
Oct 30 06:01:55.004652 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:54.998791    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.006869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.006834    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-metrics-certs\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.007150 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.007110    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lcz6\" (UniqueName: \"kubernetes.io/projected/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-kube-api-access-4lcz6\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.010110 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.010082    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-maas-api-tls\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.010482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.000884    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"]
Oct 30 06:01:55.087938 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.087903    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:55.112034 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.111991    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-metrics-certs\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.112245 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.112056    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lcz6\" (UniqueName: \"kubernetes.io/projected/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-kube-api-access-4lcz6\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.112245 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.112177    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-maas-api-tls\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.112367 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.112322    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.113013 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.112987    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.113757 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:01:55.113733    5066 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-serving-cert: secret "maas-api-e2e-inf-a-7df9fa-serving-cert" not found
Oct 30 06:01:55.113950 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:01:55.113938    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-maas-api-tls podName:3df0470a-ec9f-48fe-85bf-826b4ac4f6b8 nodeName:}" failed. No retries permitted until 2026-10-30 06:01:55.613917497 +0000 UTC m=+1819.359202815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-maas-api-tls") pod "maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6" (UID: "3df0470a-ec9f-48fe-85bf-826b4ac4f6b8") : secret "maas-api-e2e-inf-a-7df9fa-serving-cert" not found
Oct 30 06:01:55.118402 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.118379    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-metrics-certs\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.212628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.205403    5066 patch_prober.go:29] interesting pod/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Oct 30 06:01:55.212628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.205477    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" podUID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Oct 30 06:01:55.212628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.206318    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lcz6\" (UniqueName: \"kubernetes.io/projected/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-kube-api-access-4lcz6\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.270287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.270191    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:55.498432 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.498396    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:55.622335 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.622254    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-maas-api-tls\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.657610 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.656792    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-maas-api-tls\") pod \"maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:55.691421 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.691386    5066 patch_prober.go:29] interesting pod/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Oct 30 06:01:55.691624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.691448    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" podUID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Oct 30 06:01:55.915889 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:55.915843    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:56.237239 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:56.237201    5066 patch_prober.go:29] interesting pod/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Oct 30 06:01:56.237848 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:56.237817    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" podUID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Oct 30 06:01:56.481025 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:56.480994    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"]
Oct 30 06:01:56.701106 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:56.701074    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:56.702664 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:56.702643    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:56.703532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:56.703510    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:01:57.005326 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:57.005299    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6" event={"ID":"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8","Type":"ContainerStarted","Data":"19a3098f3feea7d3329908c58938512c85afbe1998bc37fe1edb1f212271c1ca"}
Oct 30 06:01:57.194067 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:57.190350    5066 patch_prober.go:29] interesting pod/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Oct 30 06:01:57.194067 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:57.190408    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" podUID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Oct 30 06:01:57.581908 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:57.581867    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:57.779393 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:57.779235    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:57.950066 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:57.950035    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:01:58.016173 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:58.016136    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6" event={"ID":"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8","Type":"ContainerStarted","Data":"001c087aeb50830b1a330f31bd7e2d0dc6a22c830455384d410876be1f1ea016"}
Oct 30 06:01:58.016451 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:58.016433    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:01:58.075516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:58.075460    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6" podStartSLOduration=3.564039951 podStartE2EDuration="4.075444064s" totalImagesPullingTime="511.404113ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:01:58.071806429 +0000 UTC m=+1821.817091761" watchObservedRunningTime="2026-10-30 06:01:58.075444064 +0000 UTC m=+1821.820729397"
Oct 30 06:01:58.191430 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:58.191399    5066 patch_prober.go:29] interesting pod/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Oct 30 06:01:58.191653 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:58.191451    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" podUID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Oct 30 06:01:59.195582 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:59.195550    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:59.196003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:59.195966    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:59.196729 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:59.196708    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:01:59.822103 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:01:59.822073    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:01.695062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:01.695030    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:02.553741 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:02:02.553709    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:02:03.529543 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:03.529514    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:04.031554 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:04.031527    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:02:05.231574 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:05.231544    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:07.090119 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:07.090086    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:08.458296 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.458262    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"]
Oct 30 06:02:08.463634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.463613    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.487228 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.487201    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3cc570-serving-cert\""
Oct 30 06:02:08.489943 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.489875    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"]
Oct 30 06:02:08.585533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.585493    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8zch\" (UniqueName: \"kubernetes.io/projected/7b475f41-1840-41fc-b58e-f1d623f8b6dd-kube-api-access-m8zch\") pod \"maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.585741 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.585679    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-metrics-certs\") pod \"maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.585741 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.585728    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-maas-api-tls\") pod \"maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.585828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.585788    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b475f41-1840-41fc-b58e-f1d623f8b6dd-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.595497 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.595459    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:08.687812 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.687776    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8zch\" (UniqueName: \"kubernetes.io/projected/7b475f41-1840-41fc-b58e-f1d623f8b6dd-kube-api-access-m8zch\") pod \"maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.687997 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.687888    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-metrics-certs\") pod \"maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.687997 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.687924    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-maas-api-tls\") pod \"maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.687997 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.687941    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b475f41-1840-41fc-b58e-f1d623f8b6dd-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.688662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.688638    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b475f41-1840-41fc-b58e-f1d623f8b6dd-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.692009 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.691984    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-metrics-certs\") pod \"maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.694213 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.694184    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-maas-api-tls\") pod \"maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.720429 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.720343    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8zch\" (UniqueName: \"kubernetes.io/projected/7b475f41-1840-41fc-b58e-f1d623f8b6dd-kube-api-access-m8zch\") pod \"maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.782971 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.782939    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:08.941765 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:08.941736    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:09.101366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:09.101335    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"]
Oct 30 06:02:09.102701 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:02:09.102666    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b475f41_1840_41fc_b58e_f1d623f8b6dd.slice/crio-566cc12c1e3d9364ba85a9084bcec1d0bbe9877a302c1dd881c3393f4dfd07ad WatchSource:0}: Error finding container 566cc12c1e3d9364ba85a9084bcec1d0bbe9877a302c1dd881c3393f4dfd07ad: Status 404 returned error can't find the container with id 566cc12c1e3d9364ba85a9084bcec1d0bbe9877a302c1dd881c3393f4dfd07ad
Oct 30 06:02:09.173197 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:09.173163    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:10.114700 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:10.114574    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94" event={"ID":"7b475f41-1840-41fc-b58e-f1d623f8b6dd","Type":"ContainerStarted","Data":"eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7"}
Oct 30 06:02:10.115220 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:10.114716    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94" event={"ID":"7b475f41-1840-41fc-b58e-f1d623f8b6dd","Type":"ContainerStarted","Data":"566cc12c1e3d9364ba85a9084bcec1d0bbe9877a302c1dd881c3393f4dfd07ad"}
Oct 30 06:02:10.115341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:10.115304    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:10.167410 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:10.167340    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94" podStartSLOduration=1.813268119 podStartE2EDuration="2.167325683s" totalImagesPullingTime="354.057564ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:02:10.163455245 +0000 UTC m=+1833.908740578" watchObservedRunningTime="2026-10-30 06:02:10.167325683 +0000 UTC m=+1833.912611015"
Oct 30 06:02:10.853193 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:10.853163    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:12.955198 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:12.955166    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:14.829619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:14.826551    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:15.097657 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:15.097554    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:15.398623 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:15.398523    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:16.131898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:16.131866    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:02:16.204985 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:16.204932    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:16.501951 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:16.501868    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:17.541995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:17.541951    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:19.510217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:19.510181    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:21.313538 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:21.313500    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:21.483854 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:21.483820    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:23.403925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:23.403888    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:24.678520 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:24.678487    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:24.875649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:24.875569    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:25.059508 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:25.059429    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:26.850283 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:26.850247    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:28.514524 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:28.514486    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:29.261361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.261329    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"]
Oct 30 06:02:29.265862 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.265836    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.291800 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.291774    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"]
Oct 30 06:02:29.342530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.342492    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-credential-socket\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.342749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.342721    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-socket\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.342848 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.342833    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9288193c-6696-4dd0-b4a9-4b2561949e06-istiod-ca-cert\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.342976 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.342951    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-envoy\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.343026 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.343002    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-token\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.343180 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.343161    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5ktj\" (UniqueName: \"kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-kube-api-access-p5ktj\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.343228 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.343206    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-certs\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.343294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.343275    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-data\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.343337 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.343308    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-podinfo\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445087 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445049    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-socket\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445258 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445143    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9288193c-6696-4dd0-b4a9-4b2561949e06-istiod-ca-cert\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445258 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445184    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-envoy\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445258 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445199    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-token\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445325    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5ktj\" (UniqueName: \"kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-kube-api-access-p5ktj\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445361    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-certs\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445468 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445392    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-data\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445468 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445417    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-podinfo\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445583 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445497    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-socket\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445583 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445508    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-credential-socket\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445798 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445775    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-certs\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445866 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445844    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-data\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445879    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9288193c-6696-4dd0-b4a9-4b2561949e06-istiod-ca-cert\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.445947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.445893    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-credential-socket\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.447689 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.447668    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-envoy\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.448039 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.448023    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-podinfo\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.470341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.470313    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-token\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.470651 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.470627    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5ktj\" (UniqueName: \"kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-kube-api-access-p5ktj\") pod \"e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") " pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.583532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.583459    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:29.911330 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.911304    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"]
Oct 30 06:02:29.912487 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:02:29.912461    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9288193c_6696_4dd0_b4a9_4b2561949e06.slice/crio-20ef25f589dd2431b784ba2da7251dac6b0e004800b41cd02c2215df1120e3bf WatchSource:0}: Error finding container 20ef25f589dd2431b784ba2da7251dac6b0e004800b41cd02c2215df1120e3bf: Status 404 returned error can't find the container with id 20ef25f589dd2431b784ba2da7251dac6b0e004800b41cd02c2215df1120e3bf
Oct 30 06:02:29.915749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.915707    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:02:29.915879 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.915804    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:02:29.915879 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.915849    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:02:29.957857 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:29.957830    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:30.121314 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:30.121282    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:30.278415 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:30.278322    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2" event={"ID":"9288193c-6696-4dd0-b4a9-4b2561949e06","Type":"ContainerStarted","Data":"30b620732453c909eac7f76ca68660cbdb1f003bd429e2af5cabb2c64814b856"}
Oct 30 06:02:30.278415 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:30.278378    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2" event={"ID":"9288193c-6696-4dd0-b4a9-4b2561949e06","Type":"ContainerStarted","Data":"20ef25f589dd2431b784ba2da7251dac6b0e004800b41cd02c2215df1120e3bf"}
Oct 30 06:02:30.333687 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:30.333634    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2" podStartSLOduration=1.333620274 podStartE2EDuration="1.333620274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:02:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:02:30.329954811 +0000 UTC m=+1854.075240145" watchObservedRunningTime="2026-10-30 06:02:30.333620274 +0000 UTC m=+1854.078905606"
Oct 30 06:02:30.584848 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:30.584760    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:31.585771 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:31.585740    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Oct 30 06:02:31.586205 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:31.585813    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2" podUID="9288193c-6696-4dd0-b4a9-4b2561949e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Oct 30 06:02:31.846671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:31.846107    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:32.020453 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:32.020404    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:32.190081 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:32.190032    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:32.588503 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:32.587994    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Oct 30 06:02:32.588503 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:32.588057    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2" podUID="9288193c-6696-4dd0-b4a9-4b2561949e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Oct 30 06:02:32.612971 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:02:32.612922    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:02:33.584331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:33.584298    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Oct 30 06:02:33.584652 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:33.584620    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2" podUID="9288193c-6696-4dd0-b4a9-4b2561949e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Oct 30 06:02:33.657580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:33.657546    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:33.861484 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:33.861403    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:34.051185 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:34.051151    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:34.583887 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:34.583849    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Oct 30 06:02:34.584064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:34.583930    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2" podUID="9288193c-6696-4dd0-b4a9-4b2561949e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Oct 30 06:02:35.517896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:35.517864    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:35.584724 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:35.584691    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Oct 30 06:02:35.584895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:35.584756    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2" podUID="9288193c-6696-4dd0-b4a9-4b2561949e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Oct 30 06:02:36.589494 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:36.589465    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:36.589951 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:36.589932    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:36.590735 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:36.590719    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:02:37.354101 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:37.354066    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:39.034030 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:39.033998    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:39.192807 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:39.192775    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:40.740578 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:40.740541    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:42.334751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:42.334715    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:43.809612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:43.805087    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:45.054291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:45.054251    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"]
Oct 30 06:02:45.062179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:45.062129    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c06f23b7-sgwb5"]
Oct 30 06:02:45.344915 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:45.344827    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:46.404982 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:46.404951    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ca003b8-b0bf-4f21-96c4-7651d99e9db6" path="/var/lib/kubelet/pods/1ca003b8-b0bf-4f21-96c4-7651d99e9db6/volumes"
Oct 30 06:02:47.258728 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:47.258689    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:49.462731 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:49.462693    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:49.743360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:49.743273    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:50.794837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:50.794800    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:51.308617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:51.301211    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:51.477464 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:51.477434    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:51.772468 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:51.772432    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:51.966616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:51.966265    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:53.228547 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:53.228508    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:54.763833 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:54.763800    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:56.304547 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:56.304488    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:58.131691 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:58.130437    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:02:59.862093 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:02:59.862004    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:00.485171 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:00.485136    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:00.657396 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:00.657362    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:01.524645 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:01.524552    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:01.895631 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:01.895535    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:02.554564 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:03:02.554529    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:03:02.913536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:02.913471    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:03.112409 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:03.112373    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:03.410612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:03.407551    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:03.613173 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:03.613104    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:03.637959 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:03.637921    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"]
Oct 30 06:03:03.650951 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:03.650919    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-84dc9d-xm4sp"]
Oct 30 06:03:03.777782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:03.777683    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:04.209582 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:04.209551    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:04.403950 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:04.403917    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b54e67a7-ee65-42d7-8496-ca62d710a0e1" path="/var/lib/kubelet/pods/b54e67a7-ee65-42d7-8496-ca62d710a0e1/volumes"
Oct 30 06:03:05.628924 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:05.628840    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:07.147427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:07.147397    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:08.651825 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:08.651792    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:09.357326 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:09.357289    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"]
Oct 30 06:03:09.357568 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:09.357544    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy" containerID="cri-o://91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5" gracePeriod=30
Oct 30 06:03:10.314519 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:10.314472    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:11.172251 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:11.172187    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:11.344962 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:11.344862    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:11.520465 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:11.520372    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:12.113122 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:12.113082    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:13.949704 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:13.949664    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:14.651063 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.651034    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 06:03:14.734492 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.734459    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"]
Oct 30 06:03:14.735619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.735573    5066 cpu_manager.go:414] "containerMap: removing container" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy"
Oct 30 06:03:14.735715 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.735651    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy"
Oct 30 06:03:14.735867 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.735853    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerName="istio-proxy"
Oct 30 06:03:14.740363 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.740343    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:14.748917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.748889    5066 generic.go:356] "Generic (PLEG): container finished" podID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" containerID="91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5" exitCode=0
Oct 30 06:03:14.749044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.749023    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"
Oct 30 06:03:14.749123 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.749038    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" event={"ID":"b718d3d8-55d8-45cc-82e2-553b3e2e1163","Type":"ContainerDied","Data":"91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5"}
Oct 30 06:03:14.749123 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.749094    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b" event={"ID":"b718d3d8-55d8-45cc-82e2-553b3e2e1163","Type":"ContainerDied","Data":"557bc42a1ee765d64e49151eedfaa4d57d96a6539cb743fdddc3dd9d306ad2cd"}
Oct 30 06:03:14.749258 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.749132    5066 scope.go:122] "RemoveContainer" containerID="91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5"
Oct 30 06:03:14.757662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.757576    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-7ad9ad-serving-cert\""
Oct 30 06:03:14.760047 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.760018    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"]
Oct 30 06:03:14.769789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.769767    5066 scope.go:122] "RemoveContainer" containerID="91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5"
Oct 30 06:03:14.770120 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:03:14.770100    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5\": container with ID starting with 91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5 not found: ID does not exist" containerID="91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5"
Oct 30 06:03:14.770190 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.770131    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5"} err="failed to get container status \"91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5\": rpc error: code = NotFound desc = could not find container \"91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5\": container with ID starting with 91efc277395fbe699ac7a95c5f658d73fe05c2dd02b45ea84d0e93c2578676b5 not found: ID does not exist"
Oct 30 06:03:14.802838 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.802805    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-socket\") pod \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") "
Oct 30 06:03:14.803049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.802850    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-token\" (UniqueName: \"kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-token\") pod \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") "
Oct 30 06:03:14.803049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.802932    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-kube-api-access-6tjqc\" (UniqueName: \"kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-kube-api-access-6tjqc\") pod \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") "
Oct 30 06:03:14.803049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.803038    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-certs\") pod \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") "
Oct 30 06:03:14.803235 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.803103    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istiod-ca-cert\") pod \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") "
Oct 30 06:03:14.803235 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.803129    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-data\") pod \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") "
Oct 30 06:03:14.803235 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.803149    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-socket" pod "b718d3d8-55d8-45cc-82e2-553b3e2e1163" (UID: "b718d3d8-55d8-45cc-82e2-553b3e2e1163"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:03:14.803235 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.803184    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-podinfo\") pod \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") "
Oct 30 06:03:14.803451 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.803335    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-credential-socket\") pod \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") "
Oct 30 06:03:14.803451 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.803350    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-certs" pod "b718d3d8-55d8-45cc-82e2-553b3e2e1163" (UID: "b718d3d8-55d8-45cc-82e2-553b3e2e1163"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:03:14.803562 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.803501    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-envoy\") pod \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\" (UID: \"b718d3d8-55d8-45cc-82e2-553b3e2e1163\") "
Oct 30 06:03:14.803562 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.803546    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-data" pod "b718d3d8-55d8-45cc-82e2-553b3e2e1163" (UID: "b718d3d8-55d8-45cc-82e2-553b3e2e1163"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:03:14.803893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.803868    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istiod-ca-cert" pod "b718d3d8-55d8-45cc-82e2-553b3e2e1163" (UID: "b718d3d8-55d8-45cc-82e2-553b3e2e1163"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:03:14.804572 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.804547    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:03:14.804723 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.804577    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:03:14.804723 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.804608    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:03:14.804723 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.804622    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:03:14.805005 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.804969    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-credential-socket" pod "b718d3d8-55d8-45cc-82e2-553b3e2e1163" (UID: "b718d3d8-55d8-45cc-82e2-553b3e2e1163"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:03:14.805828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.805798    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-kube-api-access-6tjqc" pod "b718d3d8-55d8-45cc-82e2-553b3e2e1163" (UID: "b718d3d8-55d8-45cc-82e2-553b3e2e1163"). InnerVolumeSpecName "kube-api-access-6tjqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:03:14.805989 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.805967    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-token" pod "b718d3d8-55d8-45cc-82e2-553b3e2e1163" (UID: "b718d3d8-55d8-45cc-82e2-553b3e2e1163"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:03:14.806096 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.806078    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-podinfo" pod "b718d3d8-55d8-45cc-82e2-553b3e2e1163" (UID: "b718d3d8-55d8-45cc-82e2-553b3e2e1163"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:03:14.806912 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.806891    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-envoy" pod "b718d3d8-55d8-45cc-82e2-553b3e2e1163" (UID: "b718d3d8-55d8-45cc-82e2-553b3e2e1163"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:03:14.906341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.906311    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-metrics-certs\") pod \"maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:14.906502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.906383    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:14.906502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.906463    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fc5x\" (UniqueName: \"kubernetes.io/projected/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-kube-api-access-6fc5x\") pod \"maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:14.906743 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.906721    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-maas-api-tls\") pod \"maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:14.906921 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.906909    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tjqc\" (UniqueName: \"kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-kube-api-access-6tjqc\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:03:14.906955 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.906926    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:03:14.906955 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.906934    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:03:14.906955 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.906944    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:03:14.906955 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:14.906952    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b718d3d8-55d8-45cc-82e2-553b3e2e1163-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:03:15.008830 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.008800    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-metrics-certs\") pod \"maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:15.009341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.008883    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:15.009341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.008933    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fc5x\" (UniqueName: \"kubernetes.io/projected/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-kube-api-access-6fc5x\") pod \"maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:15.009341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.009124    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-maas-api-tls\") pod \"maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:15.009910 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.009882    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:15.012180 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.012156    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-maas-api-tls\") pod \"maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:15.012284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.012156    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-metrics-certs\") pod \"maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:15.035544 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.035515    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fc5x\" (UniqueName: \"kubernetes.io/projected/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-kube-api-access-6fc5x\") pod \"maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:15.062282 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.062241    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:15.105021 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.104991    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"]
Oct 30 06:03:15.119232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.119198    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-xzq5b"]
Oct 30 06:03:15.470125 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.469883    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"]
Oct 30 06:03:15.477997 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.477971    5066 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 06:03:15.759041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.758945    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798" event={"ID":"2f369f02-bb75-4139-9eae-2a60f5c1ba6b","Type":"ContainerStarted","Data":"d7d65185897de58005cd15d84200142acdc1db699081c79adadeee5bebf0f836"}
Oct 30 06:03:15.820427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:15.820385    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:16.405280 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:16.405245    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b718d3d8-55d8-45cc-82e2-553b3e2e1163" path="/var/lib/kubelet/pods/b718d3d8-55d8-45cc-82e2-553b3e2e1163/volumes"
Oct 30 06:03:16.867658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:16.867480    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798" event={"ID":"2f369f02-bb75-4139-9eae-2a60f5c1ba6b","Type":"ContainerStarted","Data":"0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e"}
Oct 30 06:03:16.869680 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:16.868663    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:17.935862 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:17.935823    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:18.793532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:18.793478    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:19.673613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:19.673192    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:21.323073 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:21.323040    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:22.601387 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:22.601353    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:22.804706 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:22.804671    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:22.988559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:22.988524    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:23.286998 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:23.286892    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:23.680932 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:23.680891    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:23.846377 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:23.846347    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:23.893559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:23.893514    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:03:23.944122 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:23.944022    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798" podStartSLOduration=45.572578138 podStartE2EDuration="45.944009498s" totalImagesPullingTime="371.43136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:02:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:03:16.936248956 +0000 UTC m=+1900.681534291" watchObservedRunningTime="2026-10-30 06:03:23.944009498 +0000 UTC m=+1907.689294829"
Oct 30 06:03:25.068613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:25.067267    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:25.386522 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:25.386436    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:25.668670 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:25.649686    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:27.118399 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:27.117464    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:28.588724 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:28.588649    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:29.015329 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:29.014087    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:30.094582 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:30.094548    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:30.677029 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:30.676994    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:31.181668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:31.181636    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:32.013679 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:32.013624    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:32.341279 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:32.341181    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:32.567454 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:03:32.567415    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:03:33.986605 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:33.986550    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:35.461477 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:35.461440    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:35.761923 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:35.761838    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:36.910959 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:36.910924    5066 scope.go:122] "RemoveContainer" containerID="2c7965974a780a441527baf595ae8dae8d0f6338ddf644e7c8a36e7778dc5a4c"
Oct 30 06:03:37.084271 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:37.084236    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:37.681817 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:37.681780    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:38.004382 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:38.004266    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:38.191941 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:38.191898    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:38.521143 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:38.521074    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:39.784876 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:39.784840    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:39.957574 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:39.957528    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:40.134263 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:40.134175    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:41.676282 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:41.676243    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:43.158391 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:43.158353    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:44.929770 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:44.929729    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:45.301946 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:45.301862    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:45.464673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:45.464642    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:45.711907 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:45.707798    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:45.908163 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:45.908127    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:46.680753 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:46.680421    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:48.555099 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:48.555062    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:50.227795 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:50.227762    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:51.803484 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:51.803446    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:53.480400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:53.480355    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:55.124629 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:55.120235    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:55.356105 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:55.356066    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:56.640180 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:56.640148    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:56.812145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:56.812098    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:57.576072 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:57.576040    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:57.799827 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:57.799781    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:03:58.586570 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:03:58.586529    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:00.376021 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:00.375976    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:01.855782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:01.855706    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:02.559298 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:04:02.559256    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:04:02.766649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:02.766603    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:02.927410 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:02.927377    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:03.593192 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:03.593155    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:05.607950 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:05.607918    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:05.865956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:05.865858    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:06.069339 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:06.069304    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:07.938835 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:07.936516    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:09.140571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:09.140542    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:09.989828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:09.989789    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:10.291125 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:10.291010    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:10.539847 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:10.539803    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:11.846537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:11.846503    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:13.354616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:13.354448    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:14.840242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:14.840206    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:16.513356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:16.513306    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:18.108361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:18.108316    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:19.694778 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:19.694726    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:21.328514 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:21.328485    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:21.492496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:21.492465    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:22.913805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:22.913756    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:23.365272 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:23.365158    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:23.527483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:23.527444    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:24.571004 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:24.570971    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:26.176263 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:26.176226    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:27.724136 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:27.724101    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:29.279646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:29.279573    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:30.262862 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:30.262829    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:31.750662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:31.750634    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:31.917803 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:31.917753    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:32.137262 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:32.137184    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:32.556500 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:04:32.556462    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:04:33.411074 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:33.400155    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:34.876013 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:34.875972    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:36.606217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:36.606167    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:36.946863 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:36.946831    5066 scope.go:122] "RemoveContainer" containerID="b7918a8b8a753ee2f2e40e36a301acc257168ec2a85d0d35a13f062fc9be9f66"
Oct 30 06:04:36.957583 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:36.957561    5066 scope.go:122] "RemoveContainer" containerID="5040c5cba5ce54111a0e82468ae1fe484368c83e215c907969e65d5bb24f9637"
Oct 30 06:04:36.968366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:36.968345    5066 scope.go:122] "RemoveContainer" containerID="a312746d9ec9d417a3224806f9f4302c5b59cbc5756b726a38e8046f9fae5b65"
Oct 30 06:04:38.320433 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:38.320395    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:39.446403 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:39.446366    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:39.637388 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:39.637354    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:39.710841 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:39.710753    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"]
Oct 30 06:04:39.711117 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:39.711088    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" podUID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" containerName="istio-proxy" containerID="cri-o://b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407" gracePeriod=30
Oct 30 06:04:39.836644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:39.836561    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:40.003569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:40.003474    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:40.549897 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:40.549851    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:41.242085 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:41.215398    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:41.420806 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:41.420773    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:41.580761 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:41.580679    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:41.707189 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:41.707151    5066 patch_prober.go:29] interesting pod/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:04:41.707356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:41.707223    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" podUID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:04:42.081038 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:42.081000    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:42.263612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:42.260255    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:42.751657 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:42.751628    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:44.216662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:44.216581    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:45.040051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.040019    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:04:45.140219 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.140189    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-credential-socket\") pod \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") "
Oct 30 06:04:45.140421 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.140255    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-socket\") pod \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") "
Oct 30 06:04:45.140421 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.140322    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istiod-ca-cert\") pod \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") "
Oct 30 06:04:45.140421 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.140368    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-podinfo\") pod \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") "
Oct 30 06:04:45.140421 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.140400    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-envoy\") pod \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") "
Oct 30 06:04:45.140674 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.140519    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-credential-socket" pod "b01d6455-9151-4e8f-91e9-ccaa4fe18471" (UID: "b01d6455-9151-4e8f-91e9-ccaa4fe18471"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:04:45.140674 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.140537    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-data\") pod \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") "
Oct 30 06:04:45.140778 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.140655    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-socket" pod "b01d6455-9151-4e8f-91e9-ccaa4fe18471" (UID: "b01d6455-9151-4e8f-91e9-ccaa4fe18471"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:04:45.140778 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.140686    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-token\" (UniqueName: \"kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-token\") pod \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") "
Oct 30 06:04:45.141351 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.140901    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-data" pod "b01d6455-9151-4e8f-91e9-ccaa4fe18471" (UID: "b01d6455-9151-4e8f-91e9-ccaa4fe18471"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:04:45.141351 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.140826    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istiod-ca-cert" pod "b01d6455-9151-4e8f-91e9-ccaa4fe18471" (UID: "b01d6455-9151-4e8f-91e9-ccaa4fe18471"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:04:45.141351 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.140957    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-kube-api-access-x9qcd\" (UniqueName: \"kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-kube-api-access-x9qcd\") pod \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") "
Oct 30 06:04:45.141351 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.141004    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-certs\") pod \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\" (UID: \"b01d6455-9151-4e8f-91e9-ccaa4fe18471\") "
Oct 30 06:04:45.141612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.141364    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-certs" pod "b01d6455-9151-4e8f-91e9-ccaa4fe18471" (UID: "b01d6455-9151-4e8f-91e9-ccaa4fe18471"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:04:45.142156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.142052    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:04:45.142156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.142083    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:04:45.142156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.142097    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:04:45.142156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.142111    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:04:45.142156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.142128    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:04:45.143453 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.143426    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-kube-api-access-x9qcd" pod "b01d6455-9151-4e8f-91e9-ccaa4fe18471" (UID: "b01d6455-9151-4e8f-91e9-ccaa4fe18471"). InnerVolumeSpecName "kube-api-access-x9qcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:04:45.143638 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.143577    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-podinfo" pod "b01d6455-9151-4e8f-91e9-ccaa4fe18471" (UID: "b01d6455-9151-4e8f-91e9-ccaa4fe18471"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:04:45.143825 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.143798    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-envoy" pod "b01d6455-9151-4e8f-91e9-ccaa4fe18471" (UID: "b01d6455-9151-4e8f-91e9-ccaa4fe18471"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:04:45.143914 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.143896    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-token" pod "b01d6455-9151-4e8f-91e9-ccaa4fe18471" (UID: "b01d6455-9151-4e8f-91e9-ccaa4fe18471"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:04:45.243935 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.243849    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9qcd\" (UniqueName: \"kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-kube-api-access-x9qcd\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:04:45.243935 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.243887    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:04:45.243935 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.243902    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:04:45.243935 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.243915    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b01d6455-9151-4e8f-91e9-ccaa4fe18471-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:04:45.762245 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.762212    5066 generic.go:356] "Generic (PLEG): container finished" podID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" containerID="b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407" exitCode=0
Oct 30 06:04:45.762470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.762271    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"
Oct 30 06:04:45.762470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.762300    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" event={"ID":"b01d6455-9151-4e8f-91e9-ccaa4fe18471","Type":"ContainerDied","Data":"b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407"}
Oct 30 06:04:45.762470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.762357    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh" event={"ID":"b01d6455-9151-4e8f-91e9-ccaa4fe18471","Type":"ContainerDied","Data":"d21d9489d29ff55b5d3ee0fd495382e6282340ebff8aab42d55444d04c97e1f8"}
Oct 30 06:04:45.762470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.762382    5066 scope.go:122] "RemoveContainer" containerID="b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407"
Oct 30 06:04:45.773957 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.773935    5066 scope.go:122] "RemoveContainer" containerID="b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407"
Oct 30 06:04:45.774233 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:04:45.774211    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407\": container with ID starting with b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407 not found: ID does not exist" containerID="b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407"
Oct 30 06:04:45.774283 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.774244    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407"} err="failed to get container status \"b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407\": rpc error: code = NotFound desc = could not find container \"b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407\": container with ID starting with b64e6f0ca2a419c8d286e1792e96582abf260d800b7521595fc14431ef1f8407 not found: ID does not exist"
Oct 30 06:04:45.836841 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.836807    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"]
Oct 30 06:04:45.851061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.851030    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-de7242-openshift-default-7cf5999677-2cpxh"]
Oct 30 06:04:45.962935 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:45.962901    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:46.197563 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:46.197535    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:46.405271 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:46.405235    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" path="/var/lib/kubelet/pods/b01d6455-9151-4e8f-91e9-ccaa4fe18471/volumes"
Oct 30 06:04:47.743638 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:47.743571    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:47.940479 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:47.940429    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:49.615639 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:49.608765    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:49.793051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:49.793009    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:50.421675 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:50.421628    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:50.597927 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:50.597891    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:50.770699 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:50.770579    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:50.935691 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:50.935077    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:51.445070 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:51.445038    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:51.623945 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:51.623911    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:52.099061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:52.098987    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:52.542005 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:52.541956    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:53.261861 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:53.261812    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:54.967315 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:54.967277    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:56.291824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:56.291793    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:56.458759 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:56.458724    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:56.728505 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:56.728471    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:57.979418 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:57.979351    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:58.313285 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:58.313192    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:58.543625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:58.541829    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:04:58.917413 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:04:58.917359    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:00.924613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:00.924175    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:02.557224 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:05:02.557182    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:05:02.775496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:02.775452    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:04.391260 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:04.391214    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:06.117098 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:06.117064    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:07.662809 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:07.659335    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:09.359688 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:09.359658    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:11.219494 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:11.219384    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:12.863888 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:12.863848    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:14.425966 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:14.425908    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:16.112571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:16.112538    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:17.716113 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:17.716021    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:19.280474 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:19.280438    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:20.927921 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:20.927885    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:22.554538 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:22.554503    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:24.239940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:24.239906    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:26.028252 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:26.028214    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:27.442858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.442822    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"]
Oct 30 06:05:27.443552 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.443537    5066 cpu_manager.go:414] "containerMap: removing container" podUID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" containerName="istio-proxy"
Oct 30 06:05:27.443552 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.443552    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" containerName="istio-proxy"
Oct 30 06:05:27.443719 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.443708    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="b01d6455-9151-4e8f-91e9-ccaa4fe18471" containerName="istio-proxy"
Oct 30 06:05:27.447489 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.447473    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.490678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.490649    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-cfe14f-serving-cert\""
Oct 30 06:05:27.495317 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.495286    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"]
Oct 30 06:05:27.510681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.510640    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de87069-d3fc-46d6-965f-190c84575e91-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.510876 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.510766    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.510951 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.510908    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnvst\" (UniqueName: \"kubernetes.io/projected/2de87069-d3fc-46d6-965f-190c84575e91-kube-api-access-vnvst\") pod \"maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.511503 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.511477    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.569671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.569629    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:27.613325 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.613295    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnvst\" (UniqueName: \"kubernetes.io/projected/2de87069-d3fc-46d6-965f-190c84575e91-kube-api-access-vnvst\") pod \"maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.613504 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.613486    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.613549 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.613534    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de87069-d3fc-46d6-965f-190c84575e91-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.613617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.613602    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.614287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.614258    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de87069-d3fc-46d6-965f-190c84575e91-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.616425 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.616403    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.616621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.616561    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.638898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.638874    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnvst\" (UniqueName: \"kubernetes.io/projected/2de87069-d3fc-46d6-965f-190c84575e91-kube-api-access-vnvst\") pod \"maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:27.742964 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.742864    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:27.766366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:27.766333    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:28.094414 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:28.094383    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"]
Oct 30 06:05:28.094952 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:05:28.094923    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de87069_d3fc_46d6_965f_190c84575e91.slice/crio-c78e223511e7f2bc5d7765dd6d2bb6b625c49056fe23c26e132d6f950e28d4d1 WatchSource:0}: Error finding container c78e223511e7f2bc5d7765dd6d2bb6b625c49056fe23c26e132d6f950e28d4d1: Status 404 returned error can't find the container with id c78e223511e7f2bc5d7765dd6d2bb6b625c49056fe23c26e132d6f950e28d4d1
Oct 30 06:05:28.225702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:28.225661    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb" event={"ID":"2de87069-d3fc-46d6-965f-190c84575e91","Type":"ContainerStarted","Data":"c78e223511e7f2bc5d7765dd6d2bb6b625c49056fe23c26e132d6f950e28d4d1"}
Oct 30 06:05:28.746519 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:28.746399    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:29.240644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:29.237921    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb" event={"ID":"2de87069-d3fc-46d6-965f-190c84575e91","Type":"ContainerStarted","Data":"4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a"}
Oct 30 06:05:29.240644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:29.238021    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:29.286501 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:29.286437    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb" podStartSLOduration=1.9066448010000001 podStartE2EDuration="2.286422322s" totalImagesPullingTime="379.777521ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:05:29.284754277 +0000 UTC m=+2033.030039615" watchObservedRunningTime="2026-10-30 06:05:29.286422322 +0000 UTC m=+2033.031707648"
Oct 30 06:05:29.563242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:29.563101    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:31.322077 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.322029    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:31.363435 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.363393    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"]
Oct 30 06:05:31.363728 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.363700    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv" podUID="6792a38e-911d-4406-8c1a-30bf2bbfddb8" containerName="discovery" containerID="cri-o://6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b" gracePeriod=30
Oct 30 06:05:31.489237 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.489204    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:31.662564 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.662508    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:31.705467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.705439    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 06:05:31.809437 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.809402    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-kubeconfig\") pod \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") "
Oct 30 06:05:31.809718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.809471    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6792a38e-911d-4406-8c1a-30bf2bbfddb8-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6792a38e-911d-4406-8c1a-30bf2bbfddb8-local-certs\") pod \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") "
Oct 30 06:05:31.809718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.809513    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-ca-configmap\") pod \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") "
Oct 30 06:05:31.809718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.809612    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-dns-cert\") pod \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") "
Oct 30 06:05:31.809934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.809735    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-token\" (UniqueName: \"kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-token\") pod \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") "
Oct 30 06:05:31.809934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.809794    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-kube-api-access-989jx\" (UniqueName: \"kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-kube-api-access-989jx\") pod \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") "
Oct 30 06:05:31.809934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.809819    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-cacerts\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-cacerts\") pod \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\" (UID: \"6792a38e-911d-4406-8c1a-30bf2bbfddb8\") "
Oct 30 06:05:31.810874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.810543    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-ca-configmap" pod "6792a38e-911d-4406-8c1a-30bf2bbfddb8" (UID: "6792a38e-911d-4406-8c1a-30bf2bbfddb8"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:05:31.814184 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.813930    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6792a38e-911d-4406-8c1a-30bf2bbfddb8-local-certs" pod "6792a38e-911d-4406-8c1a-30bf2bbfddb8" (UID: "6792a38e-911d-4406-8c1a-30bf2bbfddb8"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:31.815044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.815020    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-cacerts" pod "6792a38e-911d-4406-8c1a-30bf2bbfddb8" (UID: "6792a38e-911d-4406-8c1a-30bf2bbfddb8"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:05:31.815147 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.815043    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-kubeconfig" pod "6792a38e-911d-4406-8c1a-30bf2bbfddb8" (UID: "6792a38e-911d-4406-8c1a-30bf2bbfddb8"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:05:31.815656 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.815578    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-dns-cert" pod "6792a38e-911d-4406-8c1a-30bf2bbfddb8" (UID: "6792a38e-911d-4406-8c1a-30bf2bbfddb8"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:05:31.819926 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.819898    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-kube-api-access-989jx" pod "6792a38e-911d-4406-8c1a-30bf2bbfddb8" (UID: "6792a38e-911d-4406-8c1a-30bf2bbfddb8"). InnerVolumeSpecName "kube-api-access-989jx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:31.820186 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.820163    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-token" pod "6792a38e-911d-4406-8c1a-30bf2bbfddb8" (UID: "6792a38e-911d-4406-8c1a-30bf2bbfddb8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:31.912161 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.912128    5066 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-cacerts\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:31.912161 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.912160    5066 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-kubeconfig\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:31.912375 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.912173    5066 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6792a38e-911d-4406-8c1a-30bf2bbfddb8-local-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:31.912375 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.912187    5066 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-ca-configmap\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:31.912375 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.912199    5066 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-csr-dns-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:31.912375 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.912209    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:31.912375 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:31.912222    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-989jx\" (UniqueName: \"kubernetes.io/projected/6792a38e-911d-4406-8c1a-30bf2bbfddb8-kube-api-access-989jx\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:32.263664 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:32.263616    5066 generic.go:356] "Generic (PLEG): container finished" podID="6792a38e-911d-4406-8c1a-30bf2bbfddb8" containerID="6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b" exitCode=0
Oct 30 06:05:32.263869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:32.263678    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"
Oct 30 06:05:32.263869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:32.263721    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv" event={"ID":"6792a38e-911d-4406-8c1a-30bf2bbfddb8","Type":"ContainerDied","Data":"6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b"}
Oct 30 06:05:32.263869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:32.263773    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv" event={"ID":"6792a38e-911d-4406-8c1a-30bf2bbfddb8","Type":"ContainerDied","Data":"46d6059064dd39f9e4b9423db8acc54daa45819db2db0a199ab7e3701785cc29"}
Oct 30 06:05:32.263869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:32.263796    5066 scope.go:122] "RemoveContainer" containerID="6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b"
Oct 30 06:05:32.278672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:32.277039    5066 scope.go:122] "RemoveContainer" containerID="6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b"
Oct 30 06:05:32.280925 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:05:32.280896    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b\": container with ID starting with 6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b not found: ID does not exist" containerID="6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b"
Oct 30 06:05:32.281039 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:32.280933    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b"} err="failed to get container status \"6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b\": rpc error: code = NotFound desc = could not find container \"6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b\": container with ID starting with 6e6e78990fb6fc850572f3328371f5790a1cb68d1bb5dcadb59314f1351eb76b not found: ID does not exist"
Oct 30 06:05:32.326313 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:32.326238    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:32.329711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:32.329689    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"]
Oct 30 06:05:32.343671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:32.343647    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6rrhv"]
Oct 30 06:05:32.406656 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:32.406621    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6792a38e-911d-4406-8c1a-30bf2bbfddb8" path="/var/lib/kubelet/pods/6792a38e-911d-4406-8c1a-30bf2bbfddb8/volumes"
Oct 30 06:05:32.552408 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:05:32.552331    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:05:33.029569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:33.029530    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:34.773235 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:34.773197    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:35.251217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:35.251190    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:05:36.730868 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:36.730813    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:37.004122 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:37.004042    5066 scope.go:122] "RemoveContainer" containerID="f8874f3d124647b79f1087f848a4ac4b5259f011c46552833d14de3cb2e1eef9"
Oct 30 06:05:37.014536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:37.014509    5066 scope.go:122] "RemoveContainer" containerID="d50ef2c84a6c7b65354028a8a29098a4086f9eeee115b91b2b7595aebe5e5130"
Oct 30 06:05:37.912130 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:37.912098    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"]
Oct 30 06:05:37.912813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:37.912798    5066 cpu_manager.go:414] "containerMap: removing container" podUID="6792a38e-911d-4406-8c1a-30bf2bbfddb8" containerName="discovery"
Oct 30 06:05:37.912868 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:37.912814    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6792a38e-911d-4406-8c1a-30bf2bbfddb8" containerName="discovery"
Oct 30 06:05:37.912931 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:37.912920    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="6792a38e-911d-4406-8c1a-30bf2bbfddb8" containerName="discovery"
Oct 30 06:05:37.918193 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:37.918173    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:37.936058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:37.935856    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c54211-openshift-default-dockercfg-t6b72\""
Oct 30 06:05:37.954509 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:37.954478    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"]
Oct 30 06:05:38.007190 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.007149    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-data\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.007435 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.007247    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40e57278-1ad4-4d16-b8be-fa1f992467ae-istiod-ca-cert\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.007435 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.007298    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-socket\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.007557 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.007509    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-token\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.007727 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.007688    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-podinfo\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.007850 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.007832    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-envoy\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.007918 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.007901    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6vz9\" (UniqueName: \"kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-kube-api-access-p6vz9\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.007985 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.007970    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-certs\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.008036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.008022    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-credential-socket\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.110050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.110015    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-podinfo\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.110244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.110080    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-envoy\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.110244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.110137    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6vz9\" (UniqueName: \"kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-kube-api-access-p6vz9\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.110244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.110180    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-certs\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.110244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.110220    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-credential-socket\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.110470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.110304    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-data\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.110470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.110354    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40e57278-1ad4-4d16-b8be-fa1f992467ae-istiod-ca-cert\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.110470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.110385    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-socket\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.110653 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.110538    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-token\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.110742 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.110720    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-certs\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.111026 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.110990    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-socket\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.111026 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.111000    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-data\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.111206 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.111044    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-credential-socket\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.111206 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.111159    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40e57278-1ad4-4d16-b8be-fa1f992467ae-istiod-ca-cert\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.112940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.112915    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-podinfo\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.113422 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.113404    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-envoy\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.120567 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.120543    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"]
Oct 30 06:05:38.120831 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.120801    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy" containerID="cri-o://0b1aae5feea16897fd3b9cd007d66c48a510cd04dd4fa91ceaf5fba82e935c99" gracePeriod=30
Oct 30 06:05:38.135893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.135858    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6vz9\" (UniqueName: \"kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-kube-api-access-p6vz9\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.136484 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.136467    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-token\") pod \"e2e-inf-b-c54211-openshift-default-767566b477-blgbh\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") " pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.232524 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.232440    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:38.375050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.375016    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:38.568381 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:05:38.568348    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e57278_1ad4_4d16_b8be_fa1f992467ae.slice/crio-99373b5f561bb25e93163ab07c963b6d135b9aa92dea0520f63dfb5e9a49a9e6 WatchSource:0}: Error finding container 99373b5f561bb25e93163ab07c963b6d135b9aa92dea0520f63dfb5e9a49a9e6: Status 404 returned error can't find the container with id 99373b5f561bb25e93163ab07c963b6d135b9aa92dea0520f63dfb5e9a49a9e6
Oct 30 06:05:38.568629 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.568557    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"]
Oct 30 06:05:38.571733 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.571696    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:05:38.571819 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.571782    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:05:38.571865 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.571818    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:05:38.676549 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.676511    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"]
Oct 30 06:05:38.676806 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:38.676779    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerName="istio-proxy" containerID="cri-o://e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1" gracePeriod=30
Oct 30 06:05:39.043849 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:39.043818    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"]
Oct 30 06:05:39.044291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:39.044027    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy" containerID="cri-o://b61695d7c06d95e5ea1e14961c897e91003db8140870c5e04f377826d69d8d19" gracePeriod=30
Oct 30 06:05:39.068852 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:39.068812    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"]
Oct 30 06:05:39.069086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:39.069057    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy" containerID="cri-o://aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c" gracePeriod=30
Oct 30 06:05:39.321364 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:39.321264    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" event={"ID":"40e57278-1ad4-4d16-b8be-fa1f992467ae","Type":"ContainerStarted","Data":"75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86"}
Oct 30 06:05:39.321364 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:39.321338    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" event={"ID":"40e57278-1ad4-4d16-b8be-fa1f992467ae","Type":"ContainerStarted","Data":"99373b5f561bb25e93163ab07c963b6d135b9aa92dea0520f63dfb5e9a49a9e6"}
Oct 30 06:05:39.373644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:39.373570    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" podStartSLOduration=2.373542774 podStartE2EDuration="2.373542774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:05:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:05:39.369718849 +0000 UTC m=+2043.115004181" watchObservedRunningTime="2026-10-30 06:05:39.373542774 +0000 UTC m=+2043.118828107"
Oct 30 06:05:39.800625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:39.800569    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"]
Oct 30 06:05:39.800849 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:39.800827    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" podUID="6660aa46-642d-4798-845c-e70792eaac8c" containerName="istio-proxy" containerID="cri-o://d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461" gracePeriod=30
Oct 30 06:05:39.962560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:39.962524    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"]
Oct 30 06:05:39.966891 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:39.966873    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:39.988089 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:39.985049    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-d3e7e6-serving-cert\""
Oct 30 06:05:40.002145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.002112    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"]
Oct 30 06:05:40.093970 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.093884    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:40.168981 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.161729    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-metrics-certs\") pod \"maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.168981 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.161983    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-maas-api-tls\") pod \"maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.168981 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.162081    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqsm9\" (UniqueName: \"kubernetes.io/projected/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-kube-api-access-zqsm9\") pod \"maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.168981 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.162143    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.245262 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.237294    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:40.245262 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.237469    5066 patch_prober.go:29] interesting pod/e2e-inf-b-c54211-openshift-default-767566b477-blgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Oct 30 06:05:40.245262 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.237513    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" podUID="40e57278-1ad4-4d16-b8be-fa1f992467ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Oct 30 06:05:40.264207 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.264180    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-metrics-certs\") pod \"maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.264773 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.264737    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-maas-api-tls\") pod \"maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.265033 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.265018    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqsm9\" (UniqueName: \"kubernetes.io/projected/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-kube-api-access-zqsm9\") pod \"maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.265233 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.265217    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.266092 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.266052    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.266266 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.266209    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:40.270211 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.270183    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-metrics-certs\") pod \"maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.270434 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.270414    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-maas-api-tls\") pod \"maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.291254 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.291223    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqsm9\" (UniqueName: \"kubernetes.io/projected/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-kube-api-access-zqsm9\") pod \"maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.449531 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.449498    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:40.586934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.586903    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:40.995548 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:40.995497    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"]
Oct 30 06:05:41.241686 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:41.241312    5066 patch_prober.go:29] interesting pod/e2e-inf-b-c54211-openshift-default-767566b477-blgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Oct 30 06:05:41.241686 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:41.241372    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" podUID="40e57278-1ad4-4d16-b8be-fa1f992467ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Oct 30 06:05:41.391610 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:41.384523    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8" event={"ID":"f5e2d33b-da00-43f1-8191-2a8f47eadf8c","Type":"ContainerStarted","Data":"8014ff346efdae2500f943f3ce10209e060c5d9dd37c37bde011161470cde11d"}
Oct 30 06:05:42.082082 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:42.082049    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:42.250303 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:42.249898    5066 patch_prober.go:29] interesting pod/e2e-inf-b-c54211-openshift-default-767566b477-blgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Oct 30 06:05:42.250303 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:42.249975    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" podUID="40e57278-1ad4-4d16-b8be-fa1f992467ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Oct 30 06:05:42.392246 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:42.392151    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8" event={"ID":"f5e2d33b-da00-43f1-8191-2a8f47eadf8c","Type":"ContainerStarted","Data":"4c5e9c25a3e8292d766a9498d013a837319e43066bf0db6f8244ab29f6f95994"}
Oct 30 06:05:42.392246 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:42.392218    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:43.247271 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.247234    5066 patch_prober.go:29] interesting pod/e2e-inf-b-c54211-openshift-default-767566b477-blgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Oct 30 06:05:43.249633 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.247943    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" podUID="40e57278-1ad4-4d16-b8be-fa1f992467ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Oct 30 06:05:43.410610 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.408281    5066 generic.go:356] "Generic (PLEG): container finished" podID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerID="0b1aae5feea16897fd3b9cd007d66c48a510cd04dd4fa91ceaf5fba82e935c99" exitCode=0
Oct 30 06:05:43.410610 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.408392    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" event={"ID":"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7","Type":"ContainerDied","Data":"0b1aae5feea16897fd3b9cd007d66c48a510cd04dd4fa91ceaf5fba82e935c99"}
Oct 30 06:05:43.492678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.492652    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 06:05:43.586816 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.586789    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-credential-socket\") pod \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") "
Oct 30 06:05:43.586977 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.586831    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-certs\") pod \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") "
Oct 30 06:05:43.586977 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.586911    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-socket\") pod \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") "
Oct 30 06:05:43.587112 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.587059    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-envoy\") pod \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") "
Oct 30 06:05:43.587112 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.587085    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-credential-socket" pod "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" (UID: "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:43.587219 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.587162    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-token\" (UniqueName: \"kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-token\") pod \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") "
Oct 30 06:05:43.587219 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.587191    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-certs" pod "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" (UID: "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:43.587332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.587275    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-data\") pod \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") "
Oct 30 06:05:43.587332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.587315    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istiod-ca-cert\") pod \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") "
Oct 30 06:05:43.587438 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.587345    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-podinfo\") pod \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") "
Oct 30 06:05:43.587438 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.587409    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-socket" pod "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" (UID: "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:43.587551 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.587440    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-kube-api-access-l4c64\" (UniqueName: \"kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-kube-api-access-l4c64\") pod \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\" (UID: \"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7\") "
Oct 30 06:05:43.588162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.587639    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-data" pod "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" (UID: "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:43.588162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.587941    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istiod-ca-cert" pod "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" (UID: "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:05:43.588162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.588100    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:43.588162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.588113    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:43.588162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.588122    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:43.588162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.588130    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:43.588162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.588138    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:43.589624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.589560    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-token" pod "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" (UID: "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:43.589885 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.589866    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-envoy" pod "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" (UID: "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:43.590018 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.589995    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-podinfo" pod "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" (UID: "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:05:43.590347 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.590327    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-kube-api-access-l4c64" pod "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" (UID: "d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7"). InnerVolumeSpecName "kube-api-access-l4c64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:43.617735 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.617678    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8" podStartSLOduration=4.198961665 podStartE2EDuration="4.617659639s" totalImagesPullingTime="418.697974ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:05:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:05:42.44898027 +0000 UTC m=+2046.194265600" watchObservedRunningTime="2026-10-30 06:05:43.617659639 +0000 UTC m=+2047.362944972"
Oct 30 06:05:43.658616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.657172    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:05:43.658616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.657234    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:05:43.690323 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.690219    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4c64\" (UniqueName: \"kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-kube-api-access-l4c64\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:43.690323 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.690255    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:43.690323 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.690270    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:43.690323 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.690282    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:43.850615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:43.848550    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:44.062419 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.062395    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 06:05:44.099276 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.099247    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-kube-api-access-fhfjw\" (UniqueName: \"kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-kube-api-access-fhfjw\") pod \"576c2b0b-6909-4f31-99b5-1dd353650d0d\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") "
Oct 30 06:05:44.099448 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.099295    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-certs\") pod \"576c2b0b-6909-4f31-99b5-1dd353650d0d\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") "
Oct 30 06:05:44.099448 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.099333    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-podinfo\") pod \"576c2b0b-6909-4f31-99b5-1dd353650d0d\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") "
Oct 30 06:05:44.099448 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.099382    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-socket\") pod \"576c2b0b-6909-4f31-99b5-1dd353650d0d\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") "
Oct 30 06:05:44.099768 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.099484    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-credential-socket\") pod \"576c2b0b-6909-4f31-99b5-1dd353650d0d\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") "
Oct 30 06:05:44.099768 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.099542    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-token\" (UniqueName: \"kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-token\") pod \"576c2b0b-6909-4f31-99b5-1dd353650d0d\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") "
Oct 30 06:05:44.099768 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.099645    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-envoy\") pod \"576c2b0b-6909-4f31-99b5-1dd353650d0d\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") "
Oct 30 06:05:44.099768 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.099689    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-certs" pod "576c2b0b-6909-4f31-99b5-1dd353650d0d" (UID: "576c2b0b-6909-4f31-99b5-1dd353650d0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.100022 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.099783    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-socket" pod "576c2b0b-6909-4f31-99b5-1dd353650d0d" (UID: "576c2b0b-6909-4f31-99b5-1dd353650d0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.100022 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.099805    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/576c2b0b-6909-4f31-99b5-1dd353650d0d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/576c2b0b-6909-4f31-99b5-1dd353650d0d-istiod-ca-cert\") pod \"576c2b0b-6909-4f31-99b5-1dd353650d0d\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") "
Oct 30 06:05:44.100022 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.099895    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-credential-socket" pod "576c2b0b-6909-4f31-99b5-1dd353650d0d" (UID: "576c2b0b-6909-4f31-99b5-1dd353650d0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.100022 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.099943    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-data\") pod \"576c2b0b-6909-4f31-99b5-1dd353650d0d\" (UID: \"576c2b0b-6909-4f31-99b5-1dd353650d0d\") "
Oct 30 06:05:44.100390 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.100155    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/576c2b0b-6909-4f31-99b5-1dd353650d0d-istiod-ca-cert" pod "576c2b0b-6909-4f31-99b5-1dd353650d0d" (UID: "576c2b0b-6909-4f31-99b5-1dd353650d0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:05:44.100390 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.100165    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-data" pod "576c2b0b-6909-4f31-99b5-1dd353650d0d" (UID: "576c2b0b-6909-4f31-99b5-1dd353650d0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.100993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.100969    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.101103 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.101000    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/576c2b0b-6909-4f31-99b5-1dd353650d0d-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.101103 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.101015    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.101103 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.101027    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.101103 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.101040    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.102067 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.102040    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-podinfo" pod "576c2b0b-6909-4f31-99b5-1dd353650d0d" (UID: "576c2b0b-6909-4f31-99b5-1dd353650d0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:05:44.102167 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.102088    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-envoy" pod "576c2b0b-6909-4f31-99b5-1dd353650d0d" (UID: "576c2b0b-6909-4f31-99b5-1dd353650d0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.102627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.102605    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-kube-api-access-fhfjw" pod "576c2b0b-6909-4f31-99b5-1dd353650d0d" (UID: "576c2b0b-6909-4f31-99b5-1dd353650d0d"). InnerVolumeSpecName "kube-api-access-fhfjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:44.102678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.102638    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-token" pod "576c2b0b-6909-4f31-99b5-1dd353650d0d" (UID: "576c2b0b-6909-4f31-99b5-1dd353650d0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:44.203098 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.203067    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.203098 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.203095    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.203291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.203110    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhfjw\" (UniqueName: \"kubernetes.io/projected/576c2b0b-6909-4f31-99b5-1dd353650d0d-kube-api-access-fhfjw\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.203291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.203123    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/576c2b0b-6909-4f31-99b5-1dd353650d0d-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.269522 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.268719    5066 patch_prober.go:29] interesting pod/e2e-inf-b-c54211-openshift-default-767566b477-blgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Oct 30 06:05:44.269522 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.268776    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" podUID="40e57278-1ad4-4d16-b8be-fa1f992467ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Oct 30 06:05:44.378343 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.378092    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 06:05:44.378343 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.378154    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 06:05:44.460062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.458376    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn" event={"ID":"d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7","Type":"ContainerDied","Data":"5267a353fbe1df7f2c030b9cd00148a29f9bdbdcbb5089fa12f7dd584a6a4e04"}
Oct 30 06:05:44.460062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.458448    5066 scope.go:122] "RemoveContainer" containerID="0b1aae5feea16897fd3b9cd007d66c48a510cd04dd4fa91ceaf5fba82e935c99"
Oct 30 06:05:44.460062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.458633    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"
Oct 30 06:05:44.470911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.470881    5066 generic.go:356] "Generic (PLEG): container finished" podID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerID="e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1" exitCode=0
Oct 30 06:05:44.471187 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.471168    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" event={"ID":"576c2b0b-6909-4f31-99b5-1dd353650d0d","Type":"ContainerDied","Data":"e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1"}
Oct 30 06:05:44.471390 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.471364    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct" event={"ID":"576c2b0b-6909-4f31-99b5-1dd353650d0d","Type":"ContainerDied","Data":"17459420a58f592ebe17ba6aeaeb3909f5b841b83b246ee25de6a846e0791879"}
Oct 30 06:05:44.473015 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.471539    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"
Oct 30 06:05:44.478272 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.478102    5066 generic.go:356] "Generic (PLEG): container finished" podID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerID="b61695d7c06d95e5ea1e14961c897e91003db8140870c5e04f377826d69d8d19" exitCode=0
Oct 30 06:05:44.478366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.478266    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" event={"ID":"7a0d40cd-44b7-405a-8967-7d3111e029e6","Type":"ContainerDied","Data":"b61695d7c06d95e5ea1e14961c897e91003db8140870c5e04f377826d69d8d19"}
Oct 30 06:05:44.479531 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.478813    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 06:05:44.481208 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.481056    5066 generic.go:356] "Generic (PLEG): container finished" podID="5f92a6f0-defd-4770-b587-6829676b2572" containerID="aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c" exitCode=0
Oct 30 06:05:44.481208 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.481151    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" event={"ID":"5f92a6f0-defd-4770-b587-6829676b2572","Type":"ContainerDied","Data":"aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c"}
Oct 30 06:05:44.481208 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.481181    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k" event={"ID":"5f92a6f0-defd-4770-b587-6829676b2572","Type":"ContainerDied","Data":"5a3ff0785ff99a8d7e95be516c4bf93b096ea9d1f77a1483fc9063dcf272d2e8"}
Oct 30 06:05:44.494625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.494602    5066 scope.go:122] "RemoveContainer" containerID="e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1"
Oct 30 06:05:44.571119 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.571020    5066 scope.go:122] "RemoveContainer" containerID="e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1"
Oct 30 06:05:44.571988 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:05:44.571558    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1\": container with ID starting with e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1 not found: ID does not exist" containerID="e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1"
Oct 30 06:05:44.573475 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.573447    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1"} err="failed to get container status \"e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1\": rpc error: code = NotFound desc = could not find container \"e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1\": container with ID starting with e170a5a90ebbd7033a9de6bd27838a04291898c1b7f67e451b7b218f257faad1 not found: ID does not exist"
Oct 30 06:05:44.574277 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.574257    5066 scope.go:122] "RemoveContainer" containerID="aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c"
Oct 30 06:05:44.578801 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.578780    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 06:05:44.590449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.590430    5066 scope.go:122] "RemoveContainer" containerID="aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c"
Oct 30 06:05:44.590779 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:05:44.590759    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c\": container with ID starting with aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c not found: ID does not exist" containerID="aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c"
Oct 30 06:05:44.590858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.590787    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c"} err="failed to get container status \"aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c\": rpc error: code = NotFound desc = could not find container \"aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c\": container with ID starting with aeca6e35a4a7162ec56c00d59da94d0d50491811cd596760bd9f13ce623da15c not found: ID does not exist"
Oct 30 06:05:44.597167 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.597141    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"]
Oct 30 06:05:44.606250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.606224    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-8f9ct"]
Oct 30 06:05:44.613932 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.613875    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f92a6f0-defd-4770-b587-6829676b2572-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f92a6f0-defd-4770-b587-6829676b2572-istiod-ca-cert\") pod \"5f92a6f0-defd-4770-b587-6829676b2572\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") "
Oct 30 06:05:44.614057 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.613986    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-socket\") pod \"5f92a6f0-defd-4770-b587-6829676b2572\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") "
Oct 30 06:05:44.614057 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.614046    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5f92a6f0-defd-4770-b587-6829676b2572-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f92a6f0-defd-4770-b587-6829676b2572-istio-podinfo\") pod \"5f92a6f0-defd-4770-b587-6829676b2572\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") "
Oct 30 06:05:44.614172 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.614126    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-credential-socket\") pod \"5f92a6f0-defd-4770-b587-6829676b2572\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") "
Oct 30 06:05:44.614223 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.614167    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-certs\") pod \"5f92a6f0-defd-4770-b587-6829676b2572\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") "
Oct 30 06:05:44.614223 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.614200    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-istio-token\" (UniqueName: \"kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-istio-token\") pod \"5f92a6f0-defd-4770-b587-6829676b2572\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") "
Oct 30 06:05:44.614331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.614267    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f92a6f0-defd-4770-b587-6829676b2572-istiod-ca-cert" pod "5f92a6f0-defd-4770-b587-6829676b2572" (UID: "5f92a6f0-defd-4770-b587-6829676b2572"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:05:44.614387 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.614354    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-socket" pod "5f92a6f0-defd-4770-b587-6829676b2572" (UID: "5f92a6f0-defd-4770-b587-6829676b2572"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.614447 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.614383    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-data\") pod \"5f92a6f0-defd-4770-b587-6829676b2572\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") "
Oct 30 06:05:44.614447 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.614426    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-kube-api-access-gjd8c\" (UniqueName: \"kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-kube-api-access-gjd8c\") pod \"5f92a6f0-defd-4770-b587-6829676b2572\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") "
Oct 30 06:05:44.614559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.614468    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-envoy\") pod \"5f92a6f0-defd-4770-b587-6829676b2572\" (UID: \"5f92a6f0-defd-4770-b587-6829676b2572\") "
Oct 30 06:05:44.614559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.614526    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-credential-socket" pod "5f92a6f0-defd-4770-b587-6829676b2572" (UID: "5f92a6f0-defd-4770-b587-6829676b2572"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.614715 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.614574    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-certs" pod "5f92a6f0-defd-4770-b587-6829676b2572" (UID: "5f92a6f0-defd-4770-b587-6829676b2572"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.614808 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.614787    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-data" pod "5f92a6f0-defd-4770-b587-6829676b2572" (UID: "5f92a6f0-defd-4770-b587-6829676b2572"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.615197 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.615169    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.615197 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.615196    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f92a6f0-defd-4770-b587-6829676b2572-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.615412 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.615208    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.615412 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.615220    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.615412 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.615232    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.617918 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.617891    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-istio-token" pod "5f92a6f0-defd-4770-b587-6829676b2572" (UID: "5f92a6f0-defd-4770-b587-6829676b2572"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:44.618021 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.618001    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f92a6f0-defd-4770-b587-6829676b2572-istio-podinfo" pod "5f92a6f0-defd-4770-b587-6829676b2572" (UID: "5f92a6f0-defd-4770-b587-6829676b2572"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:05:44.618178 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.618155    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-kube-api-access-gjd8c" pod "5f92a6f0-defd-4770-b587-6829676b2572" (UID: "5f92a6f0-defd-4770-b587-6829676b2572"). InnerVolumeSpecName "kube-api-access-gjd8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:44.619153 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.619127    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-envoy" pod "5f92a6f0-defd-4770-b587-6829676b2572" (UID: "5f92a6f0-defd-4770-b587-6829676b2572"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.668753 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.668719    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"]
Oct 30 06:05:44.673647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.673614    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-lvwxn"]
Oct 30 06:05:44.715995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.715961    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-podinfo\") pod \"7a0d40cd-44b7-405a-8967-7d3111e029e6\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") "
Oct 30 06:05:44.716179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.716053    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-credential-socket\") pod \"7a0d40cd-44b7-405a-8967-7d3111e029e6\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") "
Oct 30 06:05:44.716179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.716106    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-certs\") pod \"7a0d40cd-44b7-405a-8967-7d3111e029e6\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") "
Oct 30 06:05:44.716179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.716153    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-kube-api-access-bcmjx\" (UniqueName: \"kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-kube-api-access-bcmjx\") pod \"7a0d40cd-44b7-405a-8967-7d3111e029e6\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") "
Oct 30 06:05:44.716179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.716177    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-data\") pod \"7a0d40cd-44b7-405a-8967-7d3111e029e6\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") "
Oct 30 06:05:44.716397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.716243    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-envoy\") pod \"7a0d40cd-44b7-405a-8967-7d3111e029e6\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") "
Oct 30 06:05:44.716397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.716304    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-socket\") pod \"7a0d40cd-44b7-405a-8967-7d3111e029e6\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") "
Oct 30 06:05:44.716397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.716339    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a0d40cd-44b7-405a-8967-7d3111e029e6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a0d40cd-44b7-405a-8967-7d3111e029e6-istiod-ca-cert\") pod \"7a0d40cd-44b7-405a-8967-7d3111e029e6\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") "
Oct 30 06:05:44.716397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.716375    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-token\" (UniqueName: \"kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-token\") pod \"7a0d40cd-44b7-405a-8967-7d3111e029e6\" (UID: \"7a0d40cd-44b7-405a-8967-7d3111e029e6\") "
Oct 30 06:05:44.716619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.716413    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-credential-socket" pod "7a0d40cd-44b7-405a-8967-7d3111e029e6" (UID: "7a0d40cd-44b7-405a-8967-7d3111e029e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.716763 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.716740    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-data" pod "7a0d40cd-44b7-405a-8967-7d3111e029e6" (UID: "7a0d40cd-44b7-405a-8967-7d3111e029e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.717292 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.717253    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-socket" pod "7a0d40cd-44b7-405a-8967-7d3111e029e6" (UID: "7a0d40cd-44b7-405a-8967-7d3111e029e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.717580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.717551    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjd8c\" (UniqueName: \"kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-kube-api-access-gjd8c\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.717736 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.717581    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f92a6f0-defd-4770-b587-6829676b2572-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.717736 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.717691    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.717736 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.717709    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.717736 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.717723    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.717953 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.717736    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f92a6f0-defd-4770-b587-6829676b2572-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.717953 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.717771    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f92a6f0-defd-4770-b587-6829676b2572-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.718720 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.718245    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-certs" pod "7a0d40cd-44b7-405a-8967-7d3111e029e6" (UID: "7a0d40cd-44b7-405a-8967-7d3111e029e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.720142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.720109    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a0d40cd-44b7-405a-8967-7d3111e029e6-istiod-ca-cert" pod "7a0d40cd-44b7-405a-8967-7d3111e029e6" (UID: "7a0d40cd-44b7-405a-8967-7d3111e029e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:05:44.720735 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.720155    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-envoy" pod "7a0d40cd-44b7-405a-8967-7d3111e029e6" (UID: "7a0d40cd-44b7-405a-8967-7d3111e029e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:44.720735 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.720378    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-token" pod "7a0d40cd-44b7-405a-8967-7d3111e029e6" (UID: "7a0d40cd-44b7-405a-8967-7d3111e029e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:44.721653 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.721630    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-kube-api-access-bcmjx" pod "7a0d40cd-44b7-405a-8967-7d3111e029e6" (UID: "7a0d40cd-44b7-405a-8967-7d3111e029e6"). InnerVolumeSpecName "kube-api-access-bcmjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:44.722320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.722296    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-podinfo" pod "7a0d40cd-44b7-405a-8967-7d3111e029e6" (UID: "7a0d40cd-44b7-405a-8967-7d3111e029e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:05:44.819760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.819731    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcmjx\" (UniqueName: \"kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-kube-api-access-bcmjx\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.819760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.819760    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.819920 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.819769    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a0d40cd-44b7-405a-8967-7d3111e029e6-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.819920 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.819777    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.819920 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.819785    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a0d40cd-44b7-405a-8967-7d3111e029e6-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:44.819920 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:44.819792    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a0d40cd-44b7-405a-8967-7d3111e029e6-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:45.118935 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.118911    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 06:05:45.233245 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.233203    5066 patch_prober.go:29] interesting pod/e2e-inf-b-c54211-openshift-default-767566b477-blgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Oct 30 06:05:45.233444 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.233265    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" podUID="40e57278-1ad4-4d16-b8be-fa1f992467ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Oct 30 06:05:45.255884 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.255855    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-data\") pod \"6660aa46-642d-4798-845c-e70792eaac8c\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") "
Oct 30 06:05:45.256039 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256003    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-socket\") pod \"6660aa46-642d-4798-845c-e70792eaac8c\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") "
Oct 30 06:05:45.256116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256038    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6660aa46-642d-4798-845c-e70792eaac8c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6660aa46-642d-4798-845c-e70792eaac8c-istio-podinfo\") pod \"6660aa46-642d-4798-845c-e70792eaac8c\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") "
Oct 30 06:05:45.256116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256064    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-istio-token\" (UniqueName: \"kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-istio-token\") pod \"6660aa46-642d-4798-845c-e70792eaac8c\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") "
Oct 30 06:05:45.256238 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256121    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-data" pod "6660aa46-642d-4798-845c-e70792eaac8c" (UID: "6660aa46-642d-4798-845c-e70792eaac8c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:45.256238 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256153    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6660aa46-642d-4798-845c-e70792eaac8c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6660aa46-642d-4798-845c-e70792eaac8c-istiod-ca-cert\") pod \"6660aa46-642d-4798-845c-e70792eaac8c\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") "
Oct 30 06:05:45.256343 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256236    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-credential-socket\") pod \"6660aa46-642d-4798-845c-e70792eaac8c\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") "
Oct 30 06:05:45.256343 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256317    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-certs\") pod \"6660aa46-642d-4798-845c-e70792eaac8c\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") "
Oct 30 06:05:45.256343 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256317    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-socket" pod "6660aa46-642d-4798-845c-e70792eaac8c" (UID: "6660aa46-642d-4798-845c-e70792eaac8c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:45.256502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256391    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-kube-api-access-hlwvg\" (UniqueName: \"kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-kube-api-access-hlwvg\") pod \"6660aa46-642d-4798-845c-e70792eaac8c\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") "
Oct 30 06:05:45.256556 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256537    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-certs" pod "6660aa46-642d-4798-845c-e70792eaac8c" (UID: "6660aa46-642d-4798-845c-e70792eaac8c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:45.256634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256550    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-envoy\") pod \"6660aa46-642d-4798-845c-e70792eaac8c\" (UID: \"6660aa46-642d-4798-845c-e70792eaac8c\") "
Oct 30 06:05:45.256634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256572    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6660aa46-642d-4798-845c-e70792eaac8c-istiod-ca-cert" pod "6660aa46-642d-4798-845c-e70792eaac8c" (UID: "6660aa46-642d-4798-845c-e70792eaac8c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:05:45.256884 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.256860    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-credential-socket" pod "6660aa46-642d-4798-845c-e70792eaac8c" (UID: "6660aa46-642d-4798-845c-e70792eaac8c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:45.257532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.257508    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:45.257685 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.257535    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:45.257685 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.257548    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6660aa46-642d-4798-845c-e70792eaac8c-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:45.257685 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.257562    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:45.257685 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.257575    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:45.258764 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.258741    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6660aa46-642d-4798-845c-e70792eaac8c-istio-podinfo" pod "6660aa46-642d-4798-845c-e70792eaac8c" (UID: "6660aa46-642d-4798-845c-e70792eaac8c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:05:45.258958 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.258920    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-envoy" pod "6660aa46-642d-4798-845c-e70792eaac8c" (UID: "6660aa46-642d-4798-845c-e70792eaac8c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:45.259185 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.259161    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-istio-token" pod "6660aa46-642d-4798-845c-e70792eaac8c" (UID: "6660aa46-642d-4798-845c-e70792eaac8c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:45.259439 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.259419    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-kube-api-access-hlwvg" pod "6660aa46-642d-4798-845c-e70792eaac8c" (UID: "6660aa46-642d-4798-845c-e70792eaac8c"). InnerVolumeSpecName "kube-api-access-hlwvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:45.359335 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.359298    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6660aa46-642d-4798-845c-e70792eaac8c-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:45.359335 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.359337    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:45.359515 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.359356    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlwvg\" (UniqueName: \"kubernetes.io/projected/6660aa46-642d-4798-845c-e70792eaac8c-kube-api-access-hlwvg\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:45.359515 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.359371    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6660aa46-642d-4798-845c-e70792eaac8c-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:45.492003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.491912    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"
Oct 30 06:05:45.492003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.491945    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x" event={"ID":"7a0d40cd-44b7-405a-8967-7d3111e029e6","Type":"ContainerDied","Data":"0a7cc4bcbf9c64b2e2176fe008d5d3a0357e39c403e422e7e091ce43c9726ef8"}
Oct 30 06:05:45.492450 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.492012    5066 scope.go:122] "RemoveContainer" containerID="b61695d7c06d95e5ea1e14961c897e91003db8140870c5e04f377826d69d8d19"
Oct 30 06:05:45.494454 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.494430    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"
Oct 30 06:05:45.496513 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.496486    5066 generic.go:356] "Generic (PLEG): container finished" podID="6660aa46-642d-4798-845c-e70792eaac8c" containerID="d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461" exitCode=0
Oct 30 06:05:45.496644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.496630    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" event={"ID":"6660aa46-642d-4798-845c-e70792eaac8c","Type":"ContainerDied","Data":"d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461"}
Oct 30 06:05:45.496687 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.496675    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4" event={"ID":"6660aa46-642d-4798-845c-e70792eaac8c","Type":"ContainerDied","Data":"a1bbce18c791aa06de88d372d11f8b53a8e9ef55f4eac6875475b8a719627d93"}
Oct 30 06:05:45.496757 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.496743    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"
Oct 30 06:05:45.506718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.506554    5066 scope.go:122] "RemoveContainer" containerID="d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461"
Oct 30 06:05:45.516757 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.516737    5066 scope.go:122] "RemoveContainer" containerID="d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461"
Oct 30 06:05:45.517053 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:05:45.517028    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461\": container with ID starting with d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461 not found: ID does not exist" containerID="d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461"
Oct 30 06:05:45.517160 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.517058    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461"} err="failed to get container status \"d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461\": rpc error: code = NotFound desc = could not find container \"d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461\": container with ID starting with d86b7979933c3fcab0fd1418095e603e86f64314277dffa59cc217bd0d6fa461 not found: ID does not exist"
Oct 30 06:05:45.590522 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.590486    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"]
Oct 30 06:05:45.601465 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.601434    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-n5w9x"]
Oct 30 06:05:45.632121 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.632092    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"]
Oct 30 06:05:45.638382 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.638352    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-csjm4"]
Oct 30 06:05:45.691612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.691035    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:45.696458 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.695577    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"]
Oct 30 06:05:45.702666 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:45.702117    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-7pv8k"]
Oct 30 06:05:46.238708 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:46.238680    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:46.239162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:46.239143    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:46.239841 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:46.239824    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:05:46.405549 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:46.405514    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" path="/var/lib/kubelet/pods/576c2b0b-6909-4f31-99b5-1dd353650d0d/volumes"
Oct 30 06:05:46.406176 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:46.406159    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f92a6f0-defd-4770-b587-6829676b2572" path="/var/lib/kubelet/pods/5f92a6f0-defd-4770-b587-6829676b2572/volumes"
Oct 30 06:05:46.406633 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:46.406616    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6660aa46-642d-4798-845c-e70792eaac8c" path="/var/lib/kubelet/pods/6660aa46-642d-4798-845c-e70792eaac8c/volumes"
Oct 30 06:05:46.407380 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:46.407363    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" path="/var/lib/kubelet/pods/7a0d40cd-44b7-405a-8967-7d3111e029e6/volumes"
Oct 30 06:05:46.407826 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:46.407812    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" path="/var/lib/kubelet/pods/d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7/volumes"
Oct 30 06:05:47.264627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:47.263832    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:48.414537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:48.414509    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:05:48.772032 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:48.771926    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:50.427073 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:50.427037    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:52.010956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:52.010917    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:53.502203 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:53.502170    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:54.105008 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:54.104978    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"]
Oct 30 06:05:54.105209 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:54.105185    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy" containerID="cri-o://9d009bccc23a4ec402993f4193c30b55d9ae8327c27935fbb640e515e5d6a5f1" gracePeriod=30
Oct 30 06:05:55.092014 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:55.091976    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:56.695146 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:56.695120    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:58.335614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:58.331732    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:05:59.169977 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.169936    5066 patch_prober.go:29] interesting pod/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 06:05:59.170185 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.170017    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 06:05:59.680612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.680463    5066 generic.go:356] "Generic (PLEG): container finished" podID="77d0685c-e08d-4042-b2da-03139934718a" containerID="9d009bccc23a4ec402993f4193c30b55d9ae8327c27935fbb640e515e5d6a5f1" exitCode=0
Oct 30 06:05:59.680612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.680524    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" event={"ID":"77d0685c-e08d-4042-b2da-03139934718a","Type":"ContainerDied","Data":"9d009bccc23a4ec402993f4193c30b55d9ae8327c27935fbb640e515e5d6a5f1"}
Oct 30 06:05:59.680612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.680605    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72" event={"ID":"77d0685c-e08d-4042-b2da-03139934718a","Type":"ContainerDied","Data":"9a64a975a22449c736dde4c5cd9fcd5f909bad9784c638701513b207b6bb648d"}
Oct 30 06:05:59.681220 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.680634    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a64a975a22449c736dde4c5cd9fcd5f909bad9784c638701513b207b6bb648d"
Oct 30 06:05:59.694722 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.691868    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 06:05:59.810951 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.810918    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-socket\") pod \"77d0685c-e08d-4042-b2da-03139934718a\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") "
Oct 30 06:05:59.810951 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.810957    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-data\") pod \"77d0685c-e08d-4042-b2da-03139934718a\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") "
Oct 30 06:05:59.811227 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.810978    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-credential-socket\") pod \"77d0685c-e08d-4042-b2da-03139934718a\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") "
Oct 30 06:05:59.811227 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.811095    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-certs\") pod \"77d0685c-e08d-4042-b2da-03139934718a\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") "
Oct 30 06:05:59.811227 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.811208    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/77d0685c-e08d-4042-b2da-03139934718a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77d0685c-e08d-4042-b2da-03139934718a-istiod-ca-cert\") pod \"77d0685c-e08d-4042-b2da-03139934718a\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") "
Oct 30 06:05:59.811392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.811302    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-istio-token\" (UniqueName: \"kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-istio-token\") pod \"77d0685c-e08d-4042-b2da-03139934718a\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") "
Oct 30 06:05:59.811392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.811338    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-kube-api-access-lzns7\" (UniqueName: \"kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-kube-api-access-lzns7\") pod \"77d0685c-e08d-4042-b2da-03139934718a\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") "
Oct 30 06:05:59.811574 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.811391    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-envoy\") pod \"77d0685c-e08d-4042-b2da-03139934718a\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") "
Oct 30 06:05:59.811574 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.811416    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/77d0685c-e08d-4042-b2da-03139934718a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77d0685c-e08d-4042-b2da-03139934718a-istio-podinfo\") pod \"77d0685c-e08d-4042-b2da-03139934718a\" (UID: \"77d0685c-e08d-4042-b2da-03139934718a\") "
Oct 30 06:05:59.812438 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.812407    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-certs" pod "77d0685c-e08d-4042-b2da-03139934718a" (UID: "77d0685c-e08d-4042-b2da-03139934718a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:59.812854 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.812831    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-socket" pod "77d0685c-e08d-4042-b2da-03139934718a" (UID: "77d0685c-e08d-4042-b2da-03139934718a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:59.813097 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.813070    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d0685c-e08d-4042-b2da-03139934718a-istiod-ca-cert" pod "77d0685c-e08d-4042-b2da-03139934718a" (UID: "77d0685c-e08d-4042-b2da-03139934718a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:05:59.813324 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.813206    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-data" pod "77d0685c-e08d-4042-b2da-03139934718a" (UID: "77d0685c-e08d-4042-b2da-03139934718a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:59.813420 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.813375    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-credential-socket" pod "77d0685c-e08d-4042-b2da-03139934718a" (UID: "77d0685c-e08d-4042-b2da-03139934718a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:59.814441 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.814421    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77d0685c-e08d-4042-b2da-03139934718a-istio-podinfo" pod "77d0685c-e08d-4042-b2da-03139934718a" (UID: "77d0685c-e08d-4042-b2da-03139934718a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:05:59.816439 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.816412    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-istio-token" pod "77d0685c-e08d-4042-b2da-03139934718a" (UID: "77d0685c-e08d-4042-b2da-03139934718a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:59.817274 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.817248    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-kube-api-access-lzns7" pod "77d0685c-e08d-4042-b2da-03139934718a" (UID: "77d0685c-e08d-4042-b2da-03139934718a"). InnerVolumeSpecName "kube-api-access-lzns7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:05:59.819446 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.819421    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-envoy" pod "77d0685c-e08d-4042-b2da-03139934718a" (UID: "77d0685c-e08d-4042-b2da-03139934718a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:05:59.913988 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.913951    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:59.913988 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.913986    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77d0685c-e08d-4042-b2da-03139934718a-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:59.914204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.914001    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:59.914204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.914014    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:59.914204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.914023    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:59.914204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.914031    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77d0685c-e08d-4042-b2da-03139934718a-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:59.914204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.914038    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77d0685c-e08d-4042-b2da-03139934718a-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:59.914204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.914046    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:05:59.914204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:05:59.914056    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzns7\" (UniqueName: \"kubernetes.io/projected/77d0685c-e08d-4042-b2da-03139934718a-kube-api-access-lzns7\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:06:00.249510 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:00.249473    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:00.687724 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:00.687644    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"
Oct 30 06:06:00.761833 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:00.761797    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"]
Oct 30 06:06:00.776147 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:00.776118    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-t7j72"]
Oct 30 06:06:01.766191 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:01.766153    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:02.404054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:02.404022    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77d0685c-e08d-4042-b2da-03139934718a" path="/var/lib/kubelet/pods/77d0685c-e08d-4042-b2da-03139934718a/volumes"
Oct 30 06:06:02.554812 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:06:02.554776    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:06:03.407156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:03.407119    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:04.631330 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:04.631297    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:05.085285 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:05.085244    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:06.579436 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:06.579304    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:08.157171 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:08.157123    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:09.825013 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:09.824979    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:11.442162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:11.442119    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:13.038827 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:13.038780    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:14.929602 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:14.929545    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:16.630212 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:16.630179    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:17.639705 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.639672    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"]
Oct 30 06:06:17.640348 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640334    5066 cpu_manager.go:414] "containerMap: removing container" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerName="istio-proxy"
Oct 30 06:06:17.640392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640348    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerName="istio-proxy"
Oct 30 06:06:17.640392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640358    5066 cpu_manager.go:414] "containerMap: removing container" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy"
Oct 30 06:06:17.640392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640365    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy"
Oct 30 06:06:17.640392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640380    5066 cpu_manager.go:414] "containerMap: removing container" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy"
Oct 30 06:06:17.640392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640384    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy"
Oct 30 06:06:17.640569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640395    5066 cpu_manager.go:414] "containerMap: removing container" podUID="6660aa46-642d-4798-845c-e70792eaac8c" containerName="istio-proxy"
Oct 30 06:06:17.640569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640400    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6660aa46-642d-4798-845c-e70792eaac8c" containerName="istio-proxy"
Oct 30 06:06:17.640569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640424    5066 cpu_manager.go:414] "containerMap: removing container" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy"
Oct 30 06:06:17.640569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640428    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy"
Oct 30 06:06:17.640569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640443    5066 cpu_manager.go:414] "containerMap: removing container" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy"
Oct 30 06:06:17.640569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640448    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy"
Oct 30 06:06:17.640569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640546    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="576c2b0b-6909-4f31-99b5-1dd353650d0d" containerName="istio-proxy"
Oct 30 06:06:17.640569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640554    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f92a6f0-defd-4770-b587-6829676b2572" containerName="istio-proxy"
Oct 30 06:06:17.640569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640563    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="d345cc1b-21b6-44b4-9ca6-3a5c0acf93b7" containerName="istio-proxy"
Oct 30 06:06:17.640569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640570    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="6660aa46-642d-4798-845c-e70792eaac8c" containerName="istio-proxy"
Oct 30 06:06:17.640925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640578    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="77d0685c-e08d-4042-b2da-03139934718a" containerName="istio-proxy"
Oct 30 06:06:17.640925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.640584    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a0d40cd-44b7-405a-8967-7d3111e029e6" containerName="istio-proxy"
Oct 30 06:06:17.645421 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.645404    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.669852 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.669818    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"]
Oct 30 06:06:17.679565 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.679536    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-3cc570-openshift-default-dockercfg-svxdq\""
Oct 30 06:06:17.791854 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.791814    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-certs\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.792041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.791876    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgr8z\" (UniqueName: \"kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-kube-api-access-lgr8z\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.792041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.791971    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-envoy\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.792041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.792013    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-token\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.792256 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.792237    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-podinfo\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.792294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.792272    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istiod-ca-cert\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.792342 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.792302    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-data\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.792434 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.792416    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-credential-socket\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.792473 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.792452    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-socket\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.893543 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.893455    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-credential-socket\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.893543 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.893501    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-socket\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.893794 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.893554    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-certs\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.893794 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.893582    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgr8z\" (UniqueName: \"kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-kube-api-access-lgr8z\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.893794 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.893736    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-envoy\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.893794 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.893785    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-token\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.894012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.893987    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-podinfo\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.894012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.893992    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-credential-socket\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.894012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.894010    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istiod-ca-cert\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.894170 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.894018    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-certs\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.894170 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.894053    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-data\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.894488 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.894459    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-data\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.894807 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.894713    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istiod-ca-cert\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.894875 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.894804    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-socket\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.896321 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.896297    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-envoy\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.896552 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.896536    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-podinfo\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.921419 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.921394    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-token\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.924998 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.924975    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgr8z\" (UniqueName: \"kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-kube-api-access-lgr8z\") pod \"e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") " pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:17.960070 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:17.960039    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:18.302676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:18.302636    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"]
Oct 30 06:06:18.303123 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:06:18.303098    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0704d20f_f6e5_40b3_a61c_6afa67c7a8d6.slice/crio-f6d8bc8c8586429dc2aedd933337b964ec011f970628d64b7bc2ab182c27993f WatchSource:0}: Error finding container f6d8bc8c8586429dc2aedd933337b964ec011f970628d64b7bc2ab182c27993f: Status 404 returned error can't find the container with id f6d8bc8c8586429dc2aedd933337b964ec011f970628d64b7bc2ab182c27993f
Oct 30 06:06:18.305873 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:18.305841    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:06:18.305942 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:18.305904    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:06:18.305942 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:18.305936    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:06:18.313171 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:18.313147    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:18.514933 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:18.514899    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:18.694309 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:18.694267    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:18.829865 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:18.829812    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" event={"ID":"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6","Type":"ContainerStarted","Data":"874954af78d78d48e3794bed8e24747c19312fc019446cdec9a7953b865c560c"}
Oct 30 06:06:18.830022 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:18.829886    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" event={"ID":"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6","Type":"ContainerStarted","Data":"f6d8bc8c8586429dc2aedd933337b964ec011f970628d64b7bc2ab182c27993f"}
Oct 30 06:06:18.890834 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:18.890773    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" podStartSLOduration=1.8907525710000002 podStartE2EDuration="1.890752571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:06:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:06:18.888119116 +0000 UTC m=+2082.633404447" watchObservedRunningTime="2026-10-30 06:06:18.890752571 +0000 UTC m=+2082.636037905"
Oct 30 06:06:18.960293 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:18.960210    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:19.278170 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:19.278015    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:19.714378 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:19.714343    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:19.961870 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:19.961829    5066 patch_prober.go:29] interesting pod/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 06:06:19.962084 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:19.961884    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" podUID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 06:06:20.140663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:20.140499    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:20.961416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:20.961378    5066 patch_prober.go:29] interesting pod/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 06:06:20.961904 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:20.961449    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" podUID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 06:06:21.960819 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:21.960778    5066 patch_prober.go:29] interesting pod/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 06:06:21.961008 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:21.960839    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" podUID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 06:06:22.130897 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:22.130862    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:22.960668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:22.960635    5066 patch_prober.go:29] interesting pod/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 06:06:22.960858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:22.960692    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" podUID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 06:06:23.324703 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:23.323573    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:23.572571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:23.572527    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:23.960724 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:23.960689    5066 patch_prober.go:29] interesting pod/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 06:06:23.960937 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:23.960766    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" podUID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 06:06:24.146879 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:24.146845    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:24.960896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:24.960856    5066 patch_prober.go:29] interesting pod/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 06:06:24.961406 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:24.960925    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" podUID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 06:06:25.114622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.114574    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:25.227120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.227044    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"]
Oct 30 06:06:25.232349 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.232327    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"
Oct 30 06:06:25.252085 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.252053    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"]
Oct 30 06:06:25.262100 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.262069    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w0-03d349\""
Oct 30 06:06:25.262331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.262096    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w0-03d349-dockercfg-c4dkd\""
Oct 30 06:06:25.315946 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.315895    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"]
Oct 30 06:06:25.322238 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.322211    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"
Oct 30 06:06:25.341934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.341900    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:25.369095 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.369052    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b5b79f3-9a52-443a-be06-ab7802f9263e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt\" (UID: \"8b5b79f3-9a52-443a-be06-ab7802f9263e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"
Oct 30 06:06:25.369312 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.369199    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5fqd\" (UniqueName: \"kubernetes.io/projected/8b5b79f3-9a52-443a-be06-ab7802f9263e-kube-api-access-g5fqd\") pod \"payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt\" (UID: \"8b5b79f3-9a52-443a-be06-ab7802f9263e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"
Oct 30 06:06:25.403379 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.403340    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"]
Oct 30 06:06:25.496663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.492794    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg4dx\" (UniqueName: \"kubernetes.io/projected/92cca1a6-0f7b-40db-bb56-dd8460b2706f-kube-api-access-jg4dx\") pod \"payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm\" (UID: \"92cca1a6-0f7b-40db-bb56-dd8460b2706f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"
Oct 30 06:06:25.496663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.492913    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b5b79f3-9a52-443a-be06-ab7802f9263e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt\" (UID: \"8b5b79f3-9a52-443a-be06-ab7802f9263e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"
Oct 30 06:06:25.496663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.492994    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92cca1a6-0f7b-40db-bb56-dd8460b2706f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm\" (UID: \"92cca1a6-0f7b-40db-bb56-dd8460b2706f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"
Oct 30 06:06:25.496663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.493059    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5fqd\" (UniqueName: \"kubernetes.io/projected/8b5b79f3-9a52-443a-be06-ab7802f9263e-kube-api-access-g5fqd\") pod \"payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt\" (UID: \"8b5b79f3-9a52-443a-be06-ab7802f9263e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"
Oct 30 06:06:25.496663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.495752    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b5b79f3-9a52-443a-be06-ab7802f9263e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt\" (UID: \"8b5b79f3-9a52-443a-be06-ab7802f9263e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"
Oct 30 06:06:25.590921 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.590887    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5fqd\" (UniqueName: \"kubernetes.io/projected/8b5b79f3-9a52-443a-be06-ab7802f9263e-kube-api-access-g5fqd\") pod \"payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt\" (UID: \"8b5b79f3-9a52-443a-be06-ab7802f9263e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"
Oct 30 06:06:25.609644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.608937    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg4dx\" (UniqueName: \"kubernetes.io/projected/92cca1a6-0f7b-40db-bb56-dd8460b2706f-kube-api-access-jg4dx\") pod \"payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm\" (UID: \"92cca1a6-0f7b-40db-bb56-dd8460b2706f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"
Oct 30 06:06:25.609644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.609094    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92cca1a6-0f7b-40db-bb56-dd8460b2706f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm\" (UID: \"92cca1a6-0f7b-40db-bb56-dd8460b2706f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"
Oct 30 06:06:25.610142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.610117    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92cca1a6-0f7b-40db-bb56-dd8460b2706f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm\" (UID: \"92cca1a6-0f7b-40db-bb56-dd8460b2706f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"
Oct 30 06:06:25.641550 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.641511    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg4dx\" (UniqueName: \"kubernetes.io/projected/92cca1a6-0f7b-40db-bb56-dd8460b2706f-kube-api-access-jg4dx\") pod \"payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm\" (UID: \"92cca1a6-0f7b-40db-bb56-dd8460b2706f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"
Oct 30 06:06:25.760569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.760482    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"
Oct 30 06:06:25.868123 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:25.867864    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"
Oct 30 06:06:26.053909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:26.049808    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:26.053909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:26.050780    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:26.053909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:26.051976    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:06:26.255307 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:26.255274    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"]
Oct 30 06:06:26.256099 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:06:26.256073    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92cca1a6_0f7b_40db_bb56_dd8460b2706f.slice/crio-42c5f0731602260d0d3faa13a3cd79f2851de3c91788d6f16fbdefa58a365f16 WatchSource:0}: Error finding container 42c5f0731602260d0d3faa13a3cd79f2851de3c91788d6f16fbdefa58a365f16: Status 404 returned error can't find the container with id 42c5f0731602260d0d3faa13a3cd79f2851de3c91788d6f16fbdefa58a365f16
Oct 30 06:06:26.366174 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:26.366142    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"]
Oct 30 06:06:26.366379 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:06:26.366358    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b5b79f3_9a52_443a_be06_ab7802f9263e.slice/crio-d7b1bc83a5dbd58da134f58453986a597f8ecfb6ea4629adb5d6bde24e92669a WatchSource:0}: Error finding container d7b1bc83a5dbd58da134f58453986a597f8ecfb6ea4629adb5d6bde24e92669a: Status 404 returned error can't find the container with id d7b1bc83a5dbd58da134f58453986a597f8ecfb6ea4629adb5d6bde24e92669a
Oct 30 06:06:26.447839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:26.447780    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:26.956521 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:26.956482    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt" event={"ID":"8b5b79f3-9a52-443a-be06-ab7802f9263e","Type":"ContainerStarted","Data":"d7b1bc83a5dbd58da134f58453986a597f8ecfb6ea4629adb5d6bde24e92669a"}
Oct 30 06:06:26.959222 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:26.959190    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm" event={"ID":"92cca1a6-0f7b-40db-bb56-dd8460b2706f","Type":"ContainerStarted","Data":"42c5f0731602260d0d3faa13a3cd79f2851de3c91788d6f16fbdefa58a365f16"}
Oct 30 06:06:28.316664 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:28.316580    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:28.500001 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:28.499963    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:28.967516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:28.967485    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:28.977382 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:28.977352    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt" event={"ID":"8b5b79f3-9a52-443a-be06-ab7802f9263e","Type":"ContainerStarted","Data":"cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435"}
Oct 30 06:06:28.977572 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:28.977555    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"
Oct 30 06:06:28.979129 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:28.979107    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm" event={"ID":"92cca1a6-0f7b-40db-bb56-dd8460b2706f","Type":"ContainerStarted","Data":"ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a"}
Oct 30 06:06:28.979252 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:28.979199    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"
Oct 30 06:06:29.023906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:29.023847    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt" podStartSLOduration=1.6859457519999999 podStartE2EDuration="4.023829166s" totalImagesPullingTime="2.337883414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:06:29.021236548 +0000 UTC m=+2092.766521879" watchObservedRunningTime="2026-10-30 06:06:29.023829166 +0000 UTC m=+2092.769114503"
Oct 30 06:06:29.069415 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:29.069356    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm" podStartSLOduration=1.6267123730000002 podStartE2EDuration="4.069339696s" totalImagesPullingTime="2.442627323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:06:29.065289288 +0000 UTC m=+2092.810574621" watchObservedRunningTime="2026-10-30 06:06:29.069339696 +0000 UTC m=+2092.814625028"
Oct 30 06:06:29.558647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:29.556409    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:29.781943 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:29.781913    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:30.259299 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:30.259261    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:31.532915 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:31.532882    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:32.035945 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:32.035761    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:32.446645 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:32.446604    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:32.553638 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:06:32.553583    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:06:33.680737 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:33.680696    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:33.874904 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:33.874868    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:35.265621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:35.263407    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:35.442656 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:35.441445    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:36.003470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:36.003441    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:36.737283 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:36.737250    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:06:36.741293 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:36.741273    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:06:37.058034 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:37.057955    5066 scope.go:122] "RemoveContainer" containerID="9d009bccc23a4ec402993f4193c30b55d9ae8327c27935fbb640e515e5d6a5f1"
Oct 30 06:06:37.724247 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:37.724197    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:39.355693 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:39.355658    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:39.989311 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:39.989274    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"
Oct 30 06:06:39.989553 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:39.989535    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"
Oct 30 06:06:41.214150 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:41.214121    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:42.766018 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:42.765975    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:44.330445 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:44.330410    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:45.910063 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:45.909962    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:47.411492 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:47.411316    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:48.919681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:48.919575    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:50.841033 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:50.840993    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:52.392256 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:52.392223    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:53.959255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:53.959222    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:55.627110 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:55.625516    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:57.136255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:57.134005    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:06:58.652120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:06:58.652071    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:00.322090 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:00.322053    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:01.916232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:01.916198    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:02.553882 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:07:02.553851    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:07:03.448782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:03.448731    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:05.197041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:05.197006    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:05.772615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:05.762390    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:07.405782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.405746    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"]
Oct 30 06:07:07.411987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.411957    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:07.450560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.450531    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"]
Oct 30 06:07:07.459145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.459116    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-03d349-serving-cert\""
Oct 30 06:07:07.504055 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.504019    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:07.505863 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.504417    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skdzg\" (UniqueName: \"kubernetes.io/projected/c8ad12c2-b951-4f84-af2b-424744b62255-kube-api-access-skdzg\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:07.506238 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.506214    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:07.506457 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.506440    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ad12c2-b951-4f84-af2b-424744b62255-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:07.562739 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.562706    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:07.639615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.614558    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ad12c2-b951-4f84-af2b-424744b62255-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:07.640084 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.640035    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:07.645625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.643812    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skdzg\" (UniqueName: \"kubernetes.io/projected/c8ad12c2-b951-4f84-af2b-424744b62255-kube-api-access-skdzg\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:07.660658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.646095    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:07.661044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.624883    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ad12c2-b951-4f84-af2b-424744b62255-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:07.674620 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:07:07.653668    5066 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-serving-cert: secret "maas-api-e2e-shared-b-w0-03d349-serving-cert" not found
Oct 30 06:07:07.674931 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:07:07.674911    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-maas-api-tls podName:c8ad12c2-b951-4f84-af2b-424744b62255 nodeName:}" failed. No retries permitted until 2026-10-30 06:07:08.17487633 +0000 UTC m=+2131.920161653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-maas-api-tls") pod "maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw" (UID: "c8ad12c2-b951-4f84-af2b-424744b62255") : secret "maas-api-e2e-shared-b-w0-03d349-serving-cert" not found
Oct 30 06:07:07.682925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.678631    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:07.696211 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.696181    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skdzg\" (UniqueName: \"kubernetes.io/projected/c8ad12c2-b951-4f84-af2b-424744b62255-kube-api-access-skdzg\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:07.826323 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:07.826291    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:08.043888 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:08.043790    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:08.177206 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:08.177163    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:08.180111 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:08.180086    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:08.328532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:08.328181    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:08.708642 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:08.705844    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"]
Oct 30 06:07:08.712131 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:07:08.712095    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ad12c2_b951_4f84_af2b_424744b62255.slice/crio-4147cc2358648f556d21e2df1382318989d2875fabcede34088e9bd57e7b0291 WatchSource:0}: Error finding container 4147cc2358648f556d21e2df1382318989d2875fabcede34088e9bd57e7b0291: Status 404 returned error can't find the container with id 4147cc2358648f556d21e2df1382318989d2875fabcede34088e9bd57e7b0291
Oct 30 06:07:09.373608 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:09.371064    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw" event={"ID":"c8ad12c2-b951-4f84-af2b-424744b62255","Type":"ContainerStarted","Data":"aa5398b06228b19b1ce8272bda20a6edd1b128d402f49406e2d0267b589cdf0a"}
Oct 30 06:07:09.373608 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:09.371130    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw" event={"ID":"c8ad12c2-b951-4f84-af2b-424744b62255","Type":"ContainerStarted","Data":"4147cc2358648f556d21e2df1382318989d2875fabcede34088e9bd57e7b0291"}
Oct 30 06:07:09.373608 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:09.371608    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:09.422750 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:09.422627    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw" podStartSLOduration=2.03599821 podStartE2EDuration="2.42260869s" totalImagesPullingTime="386.61048ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:07:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:07:09.421526911 +0000 UTC m=+2133.166812246" watchObservedRunningTime="2026-10-30 06:07:09.42260869 +0000 UTC m=+2133.167894023"
Oct 30 06:07:09.540091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:09.540049    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:11.279106 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:11.279028    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:13.031912 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:13.031836    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:14.637421 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:14.637367    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:15.393418 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:15.393390    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:07:16.263684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:16.260037    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:16.535751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:16.535671    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:16.725654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:16.725568    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:18.090814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:18.090724    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:19.506016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.505986    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"]
Oct 30 06:07:19.509950 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.509930    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.537736 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.537703    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"]
Oct 30 06:07:19.546013 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.545969    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-0d0152-serving-cert\""
Oct 30 06:07:19.598012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.597981    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khq46\" (UniqueName: \"kubernetes.io/projected/ef623c5b-e3fb-4802-94ec-4dd0b445b441-kube-api-access-khq46\") pod \"maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.598222 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.598174    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-metrics-certs\") pod \"maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.598293 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.598247    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-maas-api-tls\") pod \"maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.598376 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.598349    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef623c5b-e3fb-4802-94ec-4dd0b445b441-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.640502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.640470    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:19.699961 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.699928    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-metrics-certs\") pod \"maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.700134 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.700018    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-maas-api-tls\") pod \"maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.700134 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.700045    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef623c5b-e3fb-4802-94ec-4dd0b445b441-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.700249 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.700146    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khq46\" (UniqueName: \"kubernetes.io/projected/ef623c5b-e3fb-4802-94ec-4dd0b445b441-kube-api-access-khq46\") pod \"maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.700771 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.700746    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef623c5b-e3fb-4802-94ec-4dd0b445b441-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.702642 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.702617    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-metrics-certs\") pod \"maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.702797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.702777    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-maas-api-tls\") pod \"maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.725959 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.725928    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khq46\" (UniqueName: \"kubernetes.io/projected/ef623c5b-e3fb-4802-94ec-4dd0b445b441-kube-api-access-khq46\") pod \"maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:19.802411 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.802315    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:19.831669 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:19.831583    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:20.158156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:20.158130    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"]
Oct 30 06:07:20.158283 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:07:20.158264    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef623c5b_e3fb_4802_94ec_4dd0b445b441.slice/crio-9705fb6d307fc479560e6aead98574c7d406d44977bb33798acda5d986172a76 WatchSource:0}: Error finding container 9705fb6d307fc479560e6aead98574c7d406d44977bb33798acda5d986172a76: Status 404 returned error can't find the container with id 9705fb6d307fc479560e6aead98574c7d406d44977bb33798acda5d986172a76
Oct 30 06:07:20.533313 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:20.533265    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29" event={"ID":"ef623c5b-e3fb-4802-94ec-4dd0b445b441","Type":"ContainerStarted","Data":"2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1"}
Oct 30 06:07:20.533823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:20.533340    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29" event={"ID":"ef623c5b-e3fb-4802-94ec-4dd0b445b441","Type":"ContainerStarted","Data":"9705fb6d307fc479560e6aead98574c7d406d44977bb33798acda5d986172a76"}
Oct 30 06:07:20.533823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:20.533659    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:20.599012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:20.598967    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29" podStartSLOduration=1.320222258 podStartE2EDuration="1.598951441s" totalImagesPullingTime="278.729183ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:07:20.594924494 +0000 UTC m=+2144.340209826" watchObservedRunningTime="2026-10-30 06:07:20.598951441 +0000 UTC m=+2144.344236772"
Oct 30 06:07:21.384185 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:21.384152    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:22.433602 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:22.433547    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:22.628665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:22.628574    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:23.134354 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:23.134309    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:24.750899 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:24.750865    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:26.411688 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:26.411642    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:26.548906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:26.548878    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:07:28.135349 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:28.135313    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:29.603975 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:29.603940    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:31.094355 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:31.094323    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:31.620274 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:31.620242    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:32.332525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.332485    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"]
Oct 30 06:07:32.336360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.336336    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.359354 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.359300    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-7ad9ad-openshift-default-dockercfg-kg24d\""
Oct 30 06:07:32.380227 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.380193    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"]
Oct 30 06:07:32.401806 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.401774    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-podinfo\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.401806 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.401819    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-credential-socket\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.402091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.401843    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-data\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.402091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.402034    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istiod-ca-cert\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.402091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.402068    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-token\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.402249 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.402103    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-envoy\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.402249 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.402230    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8b7g\" (UniqueName: \"kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-kube-api-access-g8b7g\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.402391 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.402375    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-socket\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.402450 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.402405    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-certs\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.503485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.503451    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istiod-ca-cert\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.503485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.503491    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-token\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.503734 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.503516    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-envoy\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.503734 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.503627    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8b7g\" (UniqueName: \"kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-kube-api-access-g8b7g\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.503809 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.503766    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-socket\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.503809 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.503797    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-certs\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.503977 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.503951    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-podinfo\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.504078 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.503990    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-credential-socket\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.504178 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.504152    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-socket\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.504242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.504153    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-data\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.504242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.504207    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-certs\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.504344 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.504300    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istiod-ca-cert\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.504501 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.504476    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-credential-socket\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.504631 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.504609    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-data\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.506181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.506161    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-envoy\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.506556 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.506535    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-podinfo\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.529965 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.529932    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-token\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.531097 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.531066    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8b7g\" (UniqueName: \"kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-kube-api-access-g8b7g\") pod \"e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") " pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.540434 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.540403    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:32.555928 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:07:32.555897    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:07:32.651299 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.651220    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:32.920785 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.920732    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:32.990385 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.990348    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"]
Oct 30 06:07:32.995025 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:07:32.994993    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d0d0d4d_3d1b_4be6_b9a0_2367550b0252.slice/crio-676c2bd144c95dd9aa5708cee9466fb35cab0a756ed377cdfe9c2355d46d56ef WatchSource:0}: Error finding container 676c2bd144c95dd9aa5708cee9466fb35cab0a756ed377cdfe9c2355d46d56ef: Status 404 returned error can't find the container with id 676c2bd144c95dd9aa5708cee9466fb35cab0a756ed377cdfe9c2355d46d56ef
Oct 30 06:07:32.998085 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.998054    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:07:32.998177 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.998126    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:07:32.998177 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:32.998166    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:07:33.652792 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:33.652741    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl" event={"ID":"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252","Type":"ContainerStarted","Data":"50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929"}
Oct 30 06:07:33.653276 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:33.652810    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl" event={"ID":"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252","Type":"ContainerStarted","Data":"676c2bd144c95dd9aa5708cee9466fb35cab0a756ed377cdfe9c2355d46d56ef"}
Oct 30 06:07:33.704895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:33.704846    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl" podStartSLOduration=1.7048300090000001 podStartE2EDuration="1.704830009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:07:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:07:33.703193117 +0000 UTC m=+2157.448478450" watchObservedRunningTime="2026-10-30 06:07:33.704830009 +0000 UTC m=+2157.450115341"
Oct 30 06:07:34.590516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.590484    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:34.663060 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.663027    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:34.675614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.674782    5066 patch_prober.go:29] interesting pod/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.140:15021/healthz/ready\": dial tcp 10.132.0.140:15021: connect: connection refused" start-of-body=
Oct 30 06:07:34.675614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.674846    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl" podUID="5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.140:15021/healthz/ready\": dial tcp 10.132.0.140:15021: connect: connection refused"
Oct 30 06:07:34.782655 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.782582    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:34.783790 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.783771    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"]
Oct 30 06:07:34.787359 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.787338    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.821838 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.821812    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-7df9fa-openshift-default-dockercfg-whvxb\""
Oct 30 06:07:34.850288 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.850189    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"]
Oct 30 06:07:34.871005 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.870962    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-certs\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.871217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.871133    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-credential-socket\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.871217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.871177    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09e2d559-d3fe-4609-ac77-04723d70ab06-istiod-ca-cert\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.871370 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.871238    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwvbt\" (UniqueName: \"kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-kube-api-access-jwvbt\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.871370 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.871334    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-socket\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.871488 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.871383    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-envoy\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.871735 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.871708    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-podinfo\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.871843 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.871810    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-data\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.871843 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.871836    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-token\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.973575 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.973540    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-certs\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.973785 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.973631    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-credential-socket\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.973785 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.973660    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09e2d559-d3fe-4609-ac77-04723d70ab06-istiod-ca-cert\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.973785 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.973728    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwvbt\" (UniqueName: \"kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-kube-api-access-jwvbt\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.973928 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.973809    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-socket\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.973928 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.973850    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-envoy\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.974027 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.973971    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-podinfo\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.974027 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.974010    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-certs\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.974142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.974073    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-data\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.974142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.974096    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-token\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.974248 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.974092    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-credential-socket\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.974488 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.974462    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09e2d559-d3fe-4609-ac77-04723d70ab06-istiod-ca-cert\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.974637 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.974527    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-data\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.974637 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.974530    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-socket\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.976510 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.976490    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-envoy\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:34.976659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:34.976642    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-podinfo\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:35.004219 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:35.004189    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-token\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:35.004408 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:35.004277    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwvbt\" (UniqueName: \"kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-kube-api-access-jwvbt\") pod \"e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") " pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:35.104537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:35.104438    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:35.475693 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:07:35.475647    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e2d559_d3fe_4609_ac77_04723d70ab06.slice/crio-468aa9e4255cd34cd4ba893b52de25827a0e0f79fae2849e18b34e67ec341d7a WatchSource:0}: Error finding container 468aa9e4255cd34cd4ba893b52de25827a0e0f79fae2849e18b34e67ec341d7a: Status 404 returned error can't find the container with id 468aa9e4255cd34cd4ba893b52de25827a0e0f79fae2849e18b34e67ec341d7a
Oct 30 06:07:35.476144 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:35.476121    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"]
Oct 30 06:07:35.478848 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:35.478815    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:07:35.478953 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:35.478883    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:07:35.478953 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:35.478914    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:07:35.651717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:35.651680    5066 patch_prober.go:29] interesting pod/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.140:15021/healthz/ready\": dial tcp 10.132.0.140:15021: connect: connection refused" start-of-body=
Oct 30 06:07:35.651929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:35.651738    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl" podUID="5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.140:15021/healthz/ready\": dial tcp 10.132.0.140:15021: connect: connection refused"
Oct 30 06:07:35.690657 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:35.690624    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" event={"ID":"09e2d559-d3fe-4609-ac77-04723d70ab06","Type":"ContainerStarted","Data":"a179d98aa61e46cf9229051e14fffce4992da27e0c70971bd94aa61a8ae9214c"}
Oct 30 06:07:35.691247 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:35.691225    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" event={"ID":"09e2d559-d3fe-4609-ac77-04723d70ab06","Type":"ContainerStarted","Data":"468aa9e4255cd34cd4ba893b52de25827a0e0f79fae2849e18b34e67ec341d7a"}
Oct 30 06:07:36.107908 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:36.107850    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:36.109221 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:36.109194    5066 patch_prober.go:29] interesting pod/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused" start-of-body=
Oct 30 06:07:36.109427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:36.109398    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused"
Oct 30 06:07:36.392671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:36.380553    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:36.673310 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:36.673272    5066 patch_prober.go:29] interesting pod/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.140:15021/healthz/ready\": dial tcp 10.132.0.140:15021: connect: connection refused" start-of-body=
Oct 30 06:07:36.673544 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:36.673331    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl" podUID="5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.140:15021/healthz/ready\": dial tcp 10.132.0.140:15021: connect: connection refused"
Oct 30 06:07:37.106670 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:37.106169    5066 patch_prober.go:29] interesting pod/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused" start-of-body=
Oct 30 06:07:37.106670 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:37.106218    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused"
Oct 30 06:07:37.663206 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:37.663172    5066 patch_prober.go:29] interesting pod/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.140:15021/healthz/ready\": dial tcp 10.132.0.140:15021: connect: connection refused" start-of-body=
Oct 30 06:07:37.663525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:37.663494    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl" podUID="5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.140:15021/healthz/ready\": dial tcp 10.132.0.140:15021: connect: connection refused"
Oct 30 06:07:38.105864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:38.105678    5066 patch_prober.go:29] interesting pod/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused" start-of-body=
Oct 30 06:07:38.105864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:38.105730    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused"
Oct 30 06:07:38.579640 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:38.571062    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:38.654786 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:38.654750    5066 patch_prober.go:29] interesting pod/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.140:15021/healthz/ready\": dial tcp 10.132.0.140:15021: connect: connection refused" start-of-body=
Oct 30 06:07:38.655074 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:38.655044    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl" podUID="5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.140:15021/healthz/ready\": dial tcp 10.132.0.140:15021: connect: connection refused"
Oct 30 06:07:39.105930 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:39.105888    5066 patch_prober.go:29] interesting pod/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused" start-of-body=
Oct 30 06:07:39.106132 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:39.105954    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused"
Oct 30 06:07:39.655191 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:39.655152    5066 patch_prober.go:29] interesting pod/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.140:15021/healthz/ready\": dial tcp 10.132.0.140:15021: connect: connection refused" start-of-body=
Oct 30 06:07:39.655800 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:39.655768    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl" podUID="5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.140:15021/healthz/ready\": dial tcp 10.132.0.140:15021: connect: connection refused"
Oct 30 06:07:40.105944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:40.105852    5066 patch_prober.go:29] interesting pod/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused" start-of-body=
Oct 30 06:07:40.105944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:40.105927    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused"
Oct 30 06:07:40.255624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:40.254038    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:40.658829 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:40.658799    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:40.659273 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:40.659252    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:40.660029 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:40.660004    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:07:40.724747 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:40.724690    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" podStartSLOduration=6.724669711 podStartE2EDuration="6.724669711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:07:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:07:35.755007133 +0000 UTC m=+2159.500292466" watchObservedRunningTime="2026-10-30 06:07:40.724669711 +0000 UTC m=+2164.469955045"
Oct 30 06:07:41.106845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:41.106745    5066 patch_prober.go:29] interesting pod/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused" start-of-body=
Oct 30 06:07:41.107038 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:41.106841    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused"
Oct 30 06:07:41.981475 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:41.981436    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:42.105938 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:42.105901    5066 patch_prober.go:29] interesting pod/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused" start-of-body=
Oct 30 06:07:42.105938 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:42.105959    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.141:15021/healthz/ready\": dial tcp 10.132.0.141:15021: connect: connection refused"
Oct 30 06:07:43.111006 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:43.110965    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:43.112560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:43.111306    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:43.112560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:43.112032    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:07:43.709148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:43.709107    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:44.481256 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:44.481219    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:44.963281 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:44.963241    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:45.625564 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:45.625527    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:47.345643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:47.345563    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:49.169120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:49.169064    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:50.865816 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:50.865760    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:52.618660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:52.616027    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:54.133886 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:54.132918    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:55.719053 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:55.719008    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:57.415094 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:57.415045    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:07:59.274920 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:07:59.274820    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:00.541572 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:00.541529    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:00.917152 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:00.916974    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:02.555047 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:08:02.555016    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:08:02.675785 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:02.675749    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:03.980226 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:03.980172    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:04.588652 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:04.584799    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:04.755524 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:04.755483    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:04.915978 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:04.915943    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:05.308889 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:05.308807    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:06.614407 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:06.614355    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:08.378803 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:08.378777    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:08.582551 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:08.582514    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:10.070855 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:10.070820    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:11.988237 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:11.988196    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:12.273904 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:12.273818    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:12.549315 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:12.549207    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:13.198191 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:13.198155    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:13.431389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:13.431347    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:13.642406 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:13.642274    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:14.031466 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:14.031421    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:15.762498 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:15.756958    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:15.949616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:15.946149    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:16.884130 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:16.884046    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"]
Oct 30 06:08:16.889061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:16.889032    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:16.915774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:16.915742    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"]
Oct 30 06:08:16.918003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:16.917977    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w0-cfe14f\"/\"default-dockercfg-qscvf\""
Oct 30 06:08:16.918003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:16.917996    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w0-cfe14f\"/\"kube-root-ca.crt\""
Oct 30 06:08:16.918169 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:16.917987    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w0-cfe14f\"/\"openshift-service-ca.crt\""
Oct 30 06:08:17.004294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.004262    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w0-cfe14f\"/\"auth-test-model-cfe14f-kserve-self-signed-certs\""
Oct 30 06:08:17.009397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.009351    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-home\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.009575 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.009427    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zzjw\" (UniqueName: \"kubernetes.io/projected/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-kube-api-access-4zzjw\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.009575 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.009533    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-model-cache\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.009746 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.009726    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-dshm\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.009822 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.009770    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tls-certs\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.009882 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.009871    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tmp-dir\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.079089 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.079054    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:17.111136 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.111096    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zzjw\" (UniqueName: \"kubernetes.io/projected/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-kube-api-access-4zzjw\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.111329 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.111204    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-model-cache\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.111393 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.111328    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-dshm\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.111393 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.111367    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tls-certs\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.111498 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.111455    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tmp-dir\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.111950 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.111603    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-home\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.111950 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.111778    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-model-cache\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.111950 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.111906    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tmp-dir\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.112192 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.111969    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-home\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.114156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.114135    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-dshm\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.114703 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.114682    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tls-certs\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.175796 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.173380    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zzjw\" (UniqueName: \"kubernetes.io/projected/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-kube-api-access-4zzjw\") pod \"auth-test-model-cfe14f-kserve-78cc66fd89-wzwll\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") " pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.219626 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.218689    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:17.651115 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.651084    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"]
Oct 30 06:08:17.652427 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:08:17.652393    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3c8e518_7e2a_4f99_8158_a7eb6932b2aa.slice/crio-6bc372c60a8690609bf963cdb5201b4389ec88a620c04f561fb61f672192ac5e WatchSource:0}: Error finding container 6bc372c60a8690609bf963cdb5201b4389ec88a620c04f561fb61f672192ac5e: Status 404 returned error can't find the container with id 6bc372c60a8690609bf963cdb5201b4389ec88a620c04f561fb61f672192ac5e
Oct 30 06:08:17.746734 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:17.746705    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:18.141795 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:18.141753    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll" event={"ID":"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa","Type":"ContainerStarted","Data":"198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8"}
Oct 30 06:08:18.142296 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:18.141819    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll" event={"ID":"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa","Type":"ContainerStarted","Data":"6bc372c60a8690609bf963cdb5201b4389ec88a620c04f561fb61f672192ac5e"}
Oct 30 06:08:18.190049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:18.189992    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll" podStartSLOduration=2.189976799 podStartE2EDuration="2.189976799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:08:18.187683709 +0000 UTC m=+2201.932969041" watchObservedRunningTime="2026-10-30 06:08:18.189976799 +0000 UTC m=+2201.935262131"
Oct 30 06:08:18.220682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:18.220649    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:19.489649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:19.489254    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:21.181364 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:21.179527    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:22.737506 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:22.737471    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:23.355452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:23.355416    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:23.544530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:23.544491    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:24.545996 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:24.545955    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:24.723046 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:24.723006    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:26.026722 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:26.026689    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:26.607289 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:26.607231    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:26.793941 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:26.790743    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:27.220058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:27.220022    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:27.233731 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:27.233699    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:27.240356 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:27.240326    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:27.314745 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:27.314716    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:08:27.409143 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:27.409097    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:28.531429 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:28.531393    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:29.794515 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:29.794486    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:29.955451 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:29.955418    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:30.115391 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:30.115281    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:31.752655 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:31.752623    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:31.928768 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:31.928733    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:32.555224 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:08:32.555188    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:08:32.659817 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:32.659787    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:33.312750 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:33.310452    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:33.872780 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:33.872749    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:34.044012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:34.043981    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:34.196170 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:34.196136    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"]
Oct 30 06:08:34.196418 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:34.196388    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerName="istio-proxy" containerID="cri-o://82c20f1f84ea1c980c48c2925daf1c7fc369b12624f7c357d0eedf1e58e17f98" gracePeriod=30
Oct 30 06:08:34.211844 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:34.211813    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"]
Oct 30 06:08:34.212091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:34.212066    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2" podUID="9288193c-6696-4dd0-b4a9-4b2561949e06" containerName="istio-proxy" containerID="cri-o://30b620732453c909eac7f76ca68660cbdb1f003bd429e2af5cabb2c64814b856" gracePeriod=30
Oct 30 06:08:34.263890 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:34.263854    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:35.347962 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.347929    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:35.855649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.855617    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"]
Oct 30 06:08:35.859522 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.859502    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:35.899030 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.898997    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d3e7e6\"/\"kube-root-ca.crt\""
Oct 30 06:08:35.899246 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.898997    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d3e7e6\"/\"openshift-service-ca.crt\""
Oct 30 06:08:35.899246 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.899147    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d3e7e6\"/\"default-dockercfg-xtx6v\""
Oct 30 06:08:35.899401 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.899385    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d3e7e6\"/\"test-model-d3e7e6-kserve-self-signed-certs\""
Oct 30 06:08:35.905222 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.905193    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"]
Oct 30 06:08:35.942642 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.942571    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:35.997713 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.997677    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-home\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:35.997924 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.997850    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-model-cache\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:35.997924 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.997882    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts7rw\" (UniqueName: \"kubernetes.io/projected/f718b500-3f4c-499a-b8a6-a6001e27e7f3-kube-api-access-ts7rw\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:35.998048 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.997983    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-dshm\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:35.998098 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.998081    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tmp-dir\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:35.998227 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:35.998202    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tls-certs\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.100017 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.099981    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tls-certs\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.100250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.100053    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-home\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.100250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.100137    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-model-cache\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.100250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.100160    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts7rw\" (UniqueName: \"kubernetes.io/projected/f718b500-3f4c-499a-b8a6-a6001e27e7f3-kube-api-access-ts7rw\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.100250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.100218    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-dshm\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.100492 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.100291    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tmp-dir\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.100961 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.100914    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-model-cache\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.100961 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.100943    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-home\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.101164 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.100997    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tmp-dir\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.103180 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.103157    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tls-certs\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.103635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.103569    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-dshm\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.163616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.162811    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts7rw\" (UniqueName: \"kubernetes.io/projected/f718b500-3f4c-499a-b8a6-a6001e27e7f3-kube-api-access-ts7rw\") pod \"test-model-d3e7e6-kserve-c5945747-g2tr9\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") " pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.178524 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.177497    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:36.592042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.591959    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:08:36.592042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.592020    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2" podUID="9288193c-6696-4dd0-b4a9-4b2561949e06" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:08:36.600616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:36.600571    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"]
Oct 30 06:08:36.600814 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:08:36.600791    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf718b500_3f4c_499a_b8a6_a6001e27e7f3.slice/crio-83e0addf443e00747c0e5b50eea97750fba1f62e9f9e5d3b6466d98362dc590a WatchSource:0}: Error finding container 83e0addf443e00747c0e5b50eea97750fba1f62e9f9e5d3b6466d98362dc590a: Status 404 returned error can't find the container with id 83e0addf443e00747c0e5b50eea97750fba1f62e9f9e5d3b6466d98362dc590a
Oct 30 06:08:37.127492 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:37.127465    5066 scope.go:122] "RemoveContainer" containerID="7cf908443bfe18bbd45724853e981f5d3c0be34aece98798b8e03b1242e93054"
Oct 30 06:08:37.401244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:37.401162    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9" event={"ID":"f718b500-3f4c-499a-b8a6-a6001e27e7f3","Type":"ContainerStarted","Data":"539848ddfb5516368856c881bf6dc79b6404673a31dff21ac33626bb6a479878"}
Oct 30 06:08:37.401244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:37.401202    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9" event={"ID":"f718b500-3f4c-499a-b8a6-a6001e27e7f3","Type":"ContainerStarted","Data":"83e0addf443e00747c0e5b50eea97750fba1f62e9f9e5d3b6466d98362dc590a"}
Oct 30 06:08:37.451093 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:37.451038    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9" podStartSLOduration=2.4510227479999998 podStartE2EDuration="2.451022748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:08:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:08:37.447721252 +0000 UTC m=+2221.193006584" watchObservedRunningTime="2026-10-30 06:08:37.451022748 +0000 UTC m=+2221.196308080"
Oct 30 06:08:37.524094 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:37.524042    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:37.633396 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:37.633354    5066 patch_prober.go:29] interesting pod/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:08:37.633837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:37.633411    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:08:38.178204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:38.178179    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:39.288111 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.288079    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:39.336489 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:08:39.336444    5066 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9288193c_6696_4dd0_b4a9_4b2561949e06.slice/crio-conmon-30b620732453c909eac7f76ca68660cbdb1f003bd429e2af5cabb2c64814b856.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9288193c_6696_4dd0_b4a9_4b2561949e06.slice/crio-30b620732453c909eac7f76ca68660cbdb1f003bd429e2af5cabb2c64814b856.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51408510_bf0c_4ad2_b776_6dff1feb777b.slice/crio-conmon-82c20f1f84ea1c980c48c2925daf1c7fc369b12624f7c357d0eedf1e58e17f98.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 06:08:39.423508 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.423082    5066 generic.go:356] "Generic (PLEG): container finished" podID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerID="82c20f1f84ea1c980c48c2925daf1c7fc369b12624f7c357d0eedf1e58e17f98" exitCode=0
Oct 30 06:08:39.423508 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.423216    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" event={"ID":"51408510-bf0c-4ad2-b776-6dff1feb777b","Type":"ContainerDied","Data":"82c20f1f84ea1c980c48c2925daf1c7fc369b12624f7c357d0eedf1e58e17f98"}
Oct 30 06:08:39.426306 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.425391    5066 generic.go:356] "Generic (PLEG): container finished" podID="9288193c-6696-4dd0-b4a9-4b2561949e06" containerID="30b620732453c909eac7f76ca68660cbdb1f003bd429e2af5cabb2c64814b856" exitCode=0
Oct 30 06:08:39.426306 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.426243    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2" event={"ID":"9288193c-6696-4dd0-b4a9-4b2561949e06","Type":"ContainerDied","Data":"30b620732453c909eac7f76ca68660cbdb1f003bd429e2af5cabb2c64814b856"}
Oct 30 06:08:39.526604 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.526565    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:08:39.563942 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.563918    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:08:39.662826 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.662792    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-credential-socket\") pod \"51408510-bf0c-4ad2-b776-6dff1feb777b\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") "
Oct 30 06:08:39.662995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.662898    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-socket\") pod \"9288193c-6696-4dd0-b4a9-4b2561949e06\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") "
Oct 30 06:08:39.662995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.662924    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51408510-bf0c-4ad2-b776-6dff1feb777b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51408510-bf0c-4ad2-b776-6dff1feb777b-istiod-ca-cert\") pod \"51408510-bf0c-4ad2-b776-6dff1feb777b\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") "
Oct 30 06:08:39.662995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.662958    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-podinfo\") pod \"9288193c-6696-4dd0-b4a9-4b2561949e06\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") "
Oct 30 06:08:39.663179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.662998    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-data\") pod \"51408510-bf0c-4ad2-b776-6dff1feb777b\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") "
Oct 30 06:08:39.663179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663050    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-envoy\") pod \"9288193c-6696-4dd0-b4a9-4b2561949e06\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") "
Oct 30 06:08:39.663179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663081    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-kube-api-access-qckmf\" (UniqueName: \"kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-kube-api-access-qckmf\") pod \"51408510-bf0c-4ad2-b776-6dff1feb777b\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") "
Oct 30 06:08:39.663179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663107    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-credential-socket" pod "51408510-bf0c-4ad2-b776-6dff1feb777b" (UID: "51408510-bf0c-4ad2-b776-6dff1feb777b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:39.663179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663147    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-token\" (UniqueName: \"kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-token\") pod \"51408510-bf0c-4ad2-b776-6dff1feb777b\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") "
Oct 30 06:08:39.663460 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663190    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-socket" pod "9288193c-6696-4dd0-b4a9-4b2561949e06" (UID: "9288193c-6696-4dd0-b4a9-4b2561949e06"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:39.663460 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663202    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9288193c-6696-4dd0-b4a9-4b2561949e06-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9288193c-6696-4dd0-b4a9-4b2561949e06-istiod-ca-cert\") pod \"9288193c-6696-4dd0-b4a9-4b2561949e06\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") "
Oct 30 06:08:39.663460 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663293    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-envoy\") pod \"51408510-bf0c-4ad2-b776-6dff1feb777b\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") "
Oct 30 06:08:39.663460 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663349    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-podinfo\") pod \"51408510-bf0c-4ad2-b776-6dff1feb777b\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") "
Oct 30 06:08:39.663460 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663377    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-data\") pod \"9288193c-6696-4dd0-b4a9-4b2561949e06\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") "
Oct 30 06:08:39.663460 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663407    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-socket\") pod \"51408510-bf0c-4ad2-b776-6dff1feb777b\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") "
Oct 30 06:08:39.663784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663464    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-credential-socket\") pod \"9288193c-6696-4dd0-b4a9-4b2561949e06\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") "
Oct 30 06:08:39.663784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663494    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-token\" (UniqueName: \"kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-token\") pod \"9288193c-6696-4dd0-b4a9-4b2561949e06\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") "
Oct 30 06:08:39.663784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663522    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-certs\") pod \"9288193c-6696-4dd0-b4a9-4b2561949e06\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") "
Oct 30 06:08:39.663784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663686    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-certs\") pod \"51408510-bf0c-4ad2-b776-6dff1feb777b\" (UID: \"51408510-bf0c-4ad2-b776-6dff1feb777b\") "
Oct 30 06:08:39.664003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663941    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9288193c-6696-4dd0-b4a9-4b2561949e06-istiod-ca-cert" pod "9288193c-6696-4dd0-b4a9-4b2561949e06" (UID: "9288193c-6696-4dd0-b4a9-4b2561949e06"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:08:39.664003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.663981    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51408510-bf0c-4ad2-b776-6dff1feb777b-istiod-ca-cert" pod "51408510-bf0c-4ad2-b776-6dff1feb777b" (UID: "51408510-bf0c-4ad2-b776-6dff1feb777b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:08:39.664109 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.664027    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-kube-api-access-p5ktj\" (UniqueName: \"kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-kube-api-access-p5ktj\") pod \"9288193c-6696-4dd0-b4a9-4b2561949e06\" (UID: \"9288193c-6696-4dd0-b4a9-4b2561949e06\") "
Oct 30 06:08:39.664899 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.664870    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-certs" pod "51408510-bf0c-4ad2-b776-6dff1feb777b" (UID: "51408510-bf0c-4ad2-b776-6dff1feb777b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:39.665029 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.664934    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.665029 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.664956    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.665029 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.664972    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51408510-bf0c-4ad2-b776-6dff1feb777b-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.665029 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.664987    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9288193c-6696-4dd0-b4a9-4b2561949e06-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.665488 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.665463    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-data" pod "51408510-bf0c-4ad2-b776-6dff1feb777b" (UID: "51408510-bf0c-4ad2-b776-6dff1feb777b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:39.665699 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.665676    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-credential-socket" pod "9288193c-6696-4dd0-b4a9-4b2561949e06" (UID: "9288193c-6696-4dd0-b4a9-4b2561949e06"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:39.665887 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.665867    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-data" pod "9288193c-6696-4dd0-b4a9-4b2561949e06" (UID: "9288193c-6696-4dd0-b4a9-4b2561949e06"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:39.666062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.666040    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-socket" pod "51408510-bf0c-4ad2-b776-6dff1feb777b" (UID: "51408510-bf0c-4ad2-b776-6dff1feb777b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:39.667652 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.667628    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-certs" pod "9288193c-6696-4dd0-b4a9-4b2561949e06" (UID: "9288193c-6696-4dd0-b4a9-4b2561949e06"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:39.669035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.669011    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-podinfo" pod "9288193c-6696-4dd0-b4a9-4b2561949e06" (UID: "9288193c-6696-4dd0-b4a9-4b2561949e06"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:08:39.669248 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.669209    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-podinfo" pod "51408510-bf0c-4ad2-b776-6dff1feb777b" (UID: "51408510-bf0c-4ad2-b776-6dff1feb777b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:08:39.669248 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.669235    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-token" pod "51408510-bf0c-4ad2-b776-6dff1feb777b" (UID: "51408510-bf0c-4ad2-b776-6dff1feb777b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:08:39.669386 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.669297    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-envoy" pod "9288193c-6696-4dd0-b4a9-4b2561949e06" (UID: "9288193c-6696-4dd0-b4a9-4b2561949e06"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:39.669678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.669631    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-envoy" pod "51408510-bf0c-4ad2-b776-6dff1feb777b" (UID: "51408510-bf0c-4ad2-b776-6dff1feb777b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:39.669678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.669635    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-kube-api-access-p5ktj" pod "9288193c-6696-4dd0-b4a9-4b2561949e06" (UID: "9288193c-6696-4dd0-b4a9-4b2561949e06"). InnerVolumeSpecName "kube-api-access-p5ktj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:08:39.669942 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.669917    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-kube-api-access-qckmf" pod "51408510-bf0c-4ad2-b776-6dff1feb777b" (UID: "51408510-bf0c-4ad2-b776-6dff1feb777b"). InnerVolumeSpecName "kube-api-access-qckmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:08:39.670504 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.670483    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-token" pod "9288193c-6696-4dd0-b4a9-4b2561949e06" (UID: "9288193c-6696-4dd0-b4a9-4b2561949e06"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:08:39.766974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.766900    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.766974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.766927    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.766974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.766936    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.766974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.766945    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.766974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.766954    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.766974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.766962    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.766974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.766970    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.767332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.766977    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5ktj\" (UniqueName: \"kubernetes.io/projected/9288193c-6696-4dd0-b4a9-4b2561949e06-kube-api-access-p5ktj\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.767332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.766996    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.767332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.767005    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.767332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.767013    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9288193c-6696-4dd0-b4a9-4b2561949e06-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.767332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.767021    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qckmf\" (UniqueName: \"kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-kube-api-access-qckmf\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.767332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.767029    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:39.767332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:39.767036    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51408510-bf0c-4ad2-b776-6dff1feb777b-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:40.357334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.357295    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"]
Oct 30 06:08:40.357911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.357494    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy" containerID="cri-o://3706b3da831df468f2cf7baa120dcee51fd012b0968d31ed9c5eab8958265120" gracePeriod=30
Oct 30 06:08:40.436382 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.436353    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"
Oct 30 06:08:40.436382 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.436371    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2" event={"ID":"9288193c-6696-4dd0-b4a9-4b2561949e06","Type":"ContainerDied","Data":"20ef25f589dd2431b784ba2da7251dac6b0e004800b41cd02c2215df1120e3bf"}
Oct 30 06:08:40.436708 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.436441    5066 scope.go:122] "RemoveContainer" containerID="30b620732453c909eac7f76ca68660cbdb1f003bd429e2af5cabb2c64814b856"
Oct 30 06:08:40.438422 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.438395    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65" event={"ID":"51408510-bf0c-4ad2-b776-6dff1feb777b","Type":"ContainerDied","Data":"7052f3101cfe619813aeb7a7f08abb06cae2e6c50631f763f0143846d98d556e"}
Oct 30 06:08:40.438534 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.438421    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"
Oct 30 06:08:40.448483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.448463    5066 scope.go:122] "RemoveContainer" containerID="82c20f1f84ea1c980c48c2925daf1c7fc369b12624f7c357d0eedf1e58e17f98"
Oct 30 06:08:40.501355 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.501278    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"]
Oct 30 06:08:40.505014 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.504992    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-r4k65"]
Oct 30 06:08:40.542506 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.542473    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"]
Oct 30 06:08:40.546021 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.545995    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-387b16-openshift-default-5bb4745897-6jpc2"]
Oct 30 06:08:40.566675 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.566642    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:40.726543 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.726509    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:40.932741 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:40.932712    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:42.404286 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:42.404251    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" path="/var/lib/kubelet/pods/51408510-bf0c-4ad2-b776-6dff1feb777b/volumes"
Oct 30 06:08:42.404889 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:42.404874    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9288193c-6696-4dd0-b4a9-4b2561949e06" path="/var/lib/kubelet/pods/9288193c-6696-4dd0-b4a9-4b2561949e06/volumes"
Oct 30 06:08:42.722910 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:42.722805    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:44.191093 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:44.191060    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:44.474138 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:44.474059    5066 patch_prober.go:29] interesting pod/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:08:44.474138 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:44.474126    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:08:45.336891 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.336843    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:45.479002 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.478976    5066 generic.go:356] "Generic (PLEG): container finished" podID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerID="3706b3da831df468f2cf7baa120dcee51fd012b0968d31ed9c5eab8958265120" exitCode=0
Oct 30 06:08:45.479122 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.479054    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" event={"ID":"cf2c8f49-dfe8-4b28-9634-c695bc74bf18","Type":"ContainerDied","Data":"3706b3da831df468f2cf7baa120dcee51fd012b0968d31ed9c5eab8958265120"}
Oct 30 06:08:45.495952 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.495857    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:45.667532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.667499    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 06:08:45.753500 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.753458    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:45.762860 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.762824    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-certs\") pod \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") "
Oct 30 06:08:45.763313 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.763285    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-certs" pod "cf2c8f49-dfe8-4b28-9634-c695bc74bf18" (UID: "cf2c8f49-dfe8-4b28-9634-c695bc74bf18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:45.763683 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.763629    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istiod-ca-cert\") pod \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") "
Oct 30 06:08:45.764407 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.764372    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-credential-socket\") pod \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") "
Oct 30 06:08:45.766188 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.764302    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istiod-ca-cert" pod "cf2c8f49-dfe8-4b28-9634-c695bc74bf18" (UID: "cf2c8f49-dfe8-4b28-9634-c695bc74bf18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:08:45.766278 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.765412    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-credential-socket" pod "cf2c8f49-dfe8-4b28-9634-c695bc74bf18" (UID: "cf2c8f49-dfe8-4b28-9634-c695bc74bf18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:45.766743 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.766720    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-data" pod "cf2c8f49-dfe8-4b28-9634-c695bc74bf18" (UID: "cf2c8f49-dfe8-4b28-9634-c695bc74bf18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:45.766865 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.766849    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-data\") pod \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") "
Oct 30 06:08:45.767044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.767020    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-token\" (UniqueName: \"kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-token\") pod \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") "
Oct 30 06:08:45.767193 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.767168    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-envoy\") pod \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") "
Oct 30 06:08:45.774483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.767202    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-kube-api-access-wkwzd\" (UniqueName: \"kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-kube-api-access-wkwzd\") pod \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") "
Oct 30 06:08:45.774483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.767238    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-socket\") pod \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") "
Oct 30 06:08:45.774483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.767331    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-podinfo\") pod \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\" (UID: \"cf2c8f49-dfe8-4b28-9634-c695bc74bf18\") "
Oct 30 06:08:45.774483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.767892    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:45.774483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.767912    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:45.774483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.767926    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:45.774483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.767940    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:45.774483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.772215    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-envoy" pod "cf2c8f49-dfe8-4b28-9634-c695bc74bf18" (UID: "cf2c8f49-dfe8-4b28-9634-c695bc74bf18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:45.796372 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.796319    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-token" pod "cf2c8f49-dfe8-4b28-9634-c695bc74bf18" (UID: "cf2c8f49-dfe8-4b28-9634-c695bc74bf18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:08:45.808267 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.808239    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-kube-api-access-wkwzd" pod "cf2c8f49-dfe8-4b28-9634-c695bc74bf18" (UID: "cf2c8f49-dfe8-4b28-9634-c695bc74bf18"). InnerVolumeSpecName "kube-api-access-wkwzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:08:45.809049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.808747    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-podinfo" pod "cf2c8f49-dfe8-4b28-9634-c695bc74bf18" (UID: "cf2c8f49-dfe8-4b28-9634-c695bc74bf18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:08:45.814901 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.814870    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-socket" pod "cf2c8f49-dfe8-4b28-9634-c695bc74bf18" (UID: "cf2c8f49-dfe8-4b28-9634-c695bc74bf18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:08:45.874965 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.874327    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:45.874965 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.874361    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkwzd\" (UniqueName: \"kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-kube-api-access-wkwzd\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:45.874965 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.874376    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:45.874965 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.874389    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:45.874965 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:45.874403    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf2c8f49-dfe8-4b28-9634-c695bc74bf18-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:46.178032 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:46.177997    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:46.190903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:46.190872    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:46.487986 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:46.487896    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"
Oct 30 06:08:46.487986 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:46.487917    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v" event={"ID":"cf2c8f49-dfe8-4b28-9634-c695bc74bf18","Type":"ContainerDied","Data":"7cde63dd77ca80bd9f11c07598da2fb70664a4be6db1c376a15a256fa7c0ec2d"}
Oct 30 06:08:46.488517 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:46.487996    5066 scope.go:122] "RemoveContainer" containerID="3706b3da831df468f2cf7baa120dcee51fd012b0968d31ed9c5eab8958265120"
Oct 30 06:08:46.502403 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:46.502377    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:08:46.555986 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:46.555951    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"]
Oct 30 06:08:46.588514 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:46.588488    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-fml6v"]
Oct 30 06:08:47.646216 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:47.646137    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:48.406987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:48.406941    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" path="/var/lib/kubelet/pods/cf2c8f49-dfe8-4b28-9634-c695bc74bf18/volumes"
Oct 30 06:08:48.436622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:48.436526    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:49.446625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:49.445953    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:51.132541 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:51.132506    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:53.015856 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:53.015800    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:54.693533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:54.693482    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:56.414422 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:56.414396    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:57.760612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:57.760546    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"]
Oct 30 06:08:57.762959 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:57.762914    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94" podUID="7b475f41-1840-41fc-b58e-f1d623f8b6dd" containerName="maas-api" containerID="cri-o://eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7" gracePeriod=30
Oct 30 06:08:57.867967 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:57.867933    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:58.456630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.451294    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:58.486576 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.486215    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:08:58.611230 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.611128    5066 generic.go:356] "Generic (PLEG): container finished" podID="7b475f41-1840-41fc-b58e-f1d623f8b6dd" containerID="eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7" exitCode=0
Oct 30 06:08:58.611230 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.611193    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"
Oct 30 06:08:58.611230 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.611214    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94" event={"ID":"7b475f41-1840-41fc-b58e-f1d623f8b6dd","Type":"ContainerDied","Data":"eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7"}
Oct 30 06:08:58.611547 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.611294    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94" event={"ID":"7b475f41-1840-41fc-b58e-f1d623f8b6dd","Type":"ContainerDied","Data":"566cc12c1e3d9364ba85a9084bcec1d0bbe9877a302c1dd881c3393f4dfd07ad"}
Oct 30 06:08:58.611547 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.611327    5066 scope.go:122] "RemoveContainer" containerID="eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7"
Oct 30 06:08:58.620096 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.620070    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b475f41-1840-41fc-b58e-f1d623f8b6dd-kube-api-access-m8zch\" (UniqueName: \"kubernetes.io/projected/7b475f41-1840-41fc-b58e-f1d623f8b6dd-kube-api-access-m8zch\") pod \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") "
Oct 30 06:08:58.620226 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.620171    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-metrics-certs\") pod \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") "
Oct 30 06:08:58.620294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.620264    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7b475f41-1840-41fc-b58e-f1d623f8b6dd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b475f41-1840-41fc-b58e-f1d623f8b6dd-openshift-service-ca\") pod \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") "
Oct 30 06:08:58.620469 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.620450    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-maas-api-tls\") pod \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\" (UID: \"7b475f41-1840-41fc-b58e-f1d623f8b6dd\") "
Oct 30 06:08:58.620640 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.620576    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b475f41-1840-41fc-b58e-f1d623f8b6dd-openshift-service-ca" pod "7b475f41-1840-41fc-b58e-f1d623f8b6dd" (UID: "7b475f41-1840-41fc-b58e-f1d623f8b6dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:08:58.621392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.621368    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b475f41-1840-41fc-b58e-f1d623f8b6dd-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:58.622556 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.622532    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b475f41-1840-41fc-b58e-f1d623f8b6dd-kube-api-access-m8zch" pod "7b475f41-1840-41fc-b58e-f1d623f8b6dd" (UID: "7b475f41-1840-41fc-b58e-f1d623f8b6dd"). InnerVolumeSpecName "kube-api-access-m8zch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:08:58.622746 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.622728    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-metrics-certs" pod "7b475f41-1840-41fc-b58e-f1d623f8b6dd" (UID: "7b475f41-1840-41fc-b58e-f1d623f8b6dd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:08:58.622855 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.622840    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-maas-api-tls" pod "7b475f41-1840-41fc-b58e-f1d623f8b6dd" (UID: "7b475f41-1840-41fc-b58e-f1d623f8b6dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:08:58.624336 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.624235    5066 scope.go:122] "RemoveContainer" containerID="eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7"
Oct 30 06:08:58.624565 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:08:58.624547    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7\": container with ID starting with eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7 not found: ID does not exist" containerID="eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7"
Oct 30 06:08:58.624689 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.624578    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7"} err="failed to get container status \"eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7\": rpc error: code = NotFound desc = could not find container \"eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7\": container with ID starting with eecd9d44b9ca56f99084dc950f771ce3b17a9a1637c534bbf6a354d1f69b2cd7 not found: ID does not exist"
Oct 30 06:08:58.723437 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.723397    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8zch\" (UniqueName: \"kubernetes.io/projected/7b475f41-1840-41fc-b58e-f1d623f8b6dd-kube-api-access-m8zch\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:58.723437 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.723434    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:58.723743 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.723449    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b475f41-1840-41fc-b58e-f1d623f8b6dd-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:08:58.787142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.787109    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:58.964751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.964719    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"]
Oct 30 06:08:58.966188 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.966138    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:08:58.970365 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:58.970343    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3cc570-6555f47d5f-kbs94"]
Oct 30 06:08:59.344505 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:08:59.344426    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:00.404347 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:00.404315    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b475f41-1840-41fc-b58e-f1d623f8b6dd" path="/var/lib/kubelet/pods/7b475f41-1840-41fc-b58e-f1d623f8b6dd/volumes"
Oct 30 06:09:00.821446 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:00.821356    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:01.683720 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:01.683681    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:02.429871 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:02.429832    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:02.555204 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:09:02.555168    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:09:03.887062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:03.887025    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:05.372944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:05.372914    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:06.019176 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:06.019102    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:06.193138 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:06.193096    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:06.322142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:06.322053    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"]
Oct 30 06:09:06.322325 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:06.322307    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" podUID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" containerName="istio-proxy" containerID="cri-o://874954af78d78d48e3794bed8e24747c19312fc019446cdec9a7953b865c560c" gracePeriod=30
Oct 30 06:09:06.359964 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:06.359930    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:06.525579 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:06.525541    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:06.727089 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:06.726979    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:08.137107 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:08.137061    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:09.619430 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:09.619385    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:11.053434 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:11.053389    5066 patch_prober.go:29] interesting pod/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:09:11.053914 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:11.053464    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" podUID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:09:11.179274 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:11.178056    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:11.722661 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:11.722631    5066 generic.go:356] "Generic (PLEG): container finished" podID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" containerID="874954af78d78d48e3794bed8e24747c19312fc019446cdec9a7953b865c560c" exitCode=0
Oct 30 06:09:11.722856 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:11.722716    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" event={"ID":"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6","Type":"ContainerDied","Data":"874954af78d78d48e3794bed8e24747c19312fc019446cdec9a7953b865c560c"}
Oct 30 06:09:12.076976 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.076944    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:09:12.154929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.154894    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-credential-socket\") pod \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") "
Oct 30 06:09:12.155119 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.154975    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-socket\") pod \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") "
Oct 30 06:09:12.155119 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.155077    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-kube-api-access-lgr8z\" (UniqueName: \"kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-kube-api-access-lgr8z\") pod \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") "
Oct 30 06:09:12.155239 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.155122    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-podinfo\") pod \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") "
Oct 30 06:09:12.155364 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.155292    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-envoy\") pod \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") "
Oct 30 06:09:12.155364 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.155297    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-socket" pod "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" (UID: "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:09:12.155364 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.155334    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-certs\") pod \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") "
Oct 30 06:09:12.155658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.155368    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-token\" (UniqueName: \"kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-token\") pod \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") "
Oct 30 06:09:12.155658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.155416    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istiod-ca-cert\") pod \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") "
Oct 30 06:09:12.155658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.155448    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-data\") pod \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\" (UID: \"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6\") "
Oct 30 06:09:12.162888 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.155930    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-credential-socket" pod "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" (UID: "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:09:12.163195 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.157457    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istiod-ca-cert" pod "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" (UID: "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:09:12.163337 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.157752    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-certs" pod "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" (UID: "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:09:12.164207 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.164178    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-kube-api-access-lgr8z" pod "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" (UID: "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6"). InnerVolumeSpecName "kube-api-access-lgr8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:09:12.166067 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.166039    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgr8z\" (UniqueName: \"kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-kube-api-access-lgr8z\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:12.166249 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.166233    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:12.166353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.166341    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:12.166443 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.166431    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:12.166542 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.166527    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:12.166957 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.166853    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-data" pod "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" (UID: "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:09:12.169232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.169191    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-envoy" pod "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" (UID: "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:09:12.169560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.169540    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-podinfo" pod "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" (UID: "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:09:12.169867 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.169847    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-token" pod "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" (UID: "0704d20f-f6e5-40b3-a61c-6afa67c7a8d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:09:12.268785 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.268743    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:12.268785 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.268777    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:12.268785 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.268789    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:12.269082 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.268801    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:12.733527 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.733444    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"
Oct 30 06:09:12.733527 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.733484    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg" event={"ID":"0704d20f-f6e5-40b3-a61c-6afa67c7a8d6","Type":"ContainerDied","Data":"f6d8bc8c8586429dc2aedd933337b964ec011f970628d64b7bc2ab182c27993f"}
Oct 30 06:09:12.733786 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.733554    5066 scope.go:122] "RemoveContainer" containerID="874954af78d78d48e3794bed8e24747c19312fc019446cdec9a7953b865c560c"
Oct 30 06:09:12.830157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.830108    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"]
Oct 30 06:09:12.834702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.834670    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-3cc570-openshift-default-7f6b9c66ff-dnfgg"]
Oct 30 06:09:12.976845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:12.976814    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:13.851999 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:13.851948    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:14.072551 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:14.072483    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:14.258171 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:14.258128    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:14.404925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:14.404866    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" path="/var/lib/kubelet/pods/0704d20f-f6e5-40b3-a61c-6afa67c7a8d6/volumes"
Oct 30 06:09:14.531178 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:14.531068    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:16.211147 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:16.211077    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:17.708909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:17.708831    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:19.480829 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:19.480787    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:20.748322 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:20.748283    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"]
Oct 30 06:09:20.748907 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:20.748541    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798" podUID="2f369f02-bb75-4139-9eae-2a60f5c1ba6b" containerName="maas-api" containerID="cri-o://0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e" gracePeriod=30
Oct 30 06:09:20.868394 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:20.868342    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:21.364159 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.364126    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:09:21.453638 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.453613    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:21.480526 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.480439    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-maas-api-tls\") pod \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") "
Oct 30 06:09:21.480713 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.480653    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-metrics-certs\") pod \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") "
Oct 30 06:09:21.480779 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.480737    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-kube-api-access-6fc5x\" (UniqueName: \"kubernetes.io/projected/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-kube-api-access-6fc5x\") pod \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") "
Oct 30 06:09:21.480947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.480929    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-openshift-service-ca\") pod \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\" (UID: \"2f369f02-bb75-4139-9eae-2a60f5c1ba6b\") "
Oct 30 06:09:21.482012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.481979    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-openshift-service-ca" pod "2f369f02-bb75-4139-9eae-2a60f5c1ba6b" (UID: "2f369f02-bb75-4139-9eae-2a60f5c1ba6b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:09:21.483644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.483580    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-metrics-certs" pod "2f369f02-bb75-4139-9eae-2a60f5c1ba6b" (UID: "2f369f02-bb75-4139-9eae-2a60f5c1ba6b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:09:21.484029 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.484007    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-kube-api-access-6fc5x" pod "2f369f02-bb75-4139-9eae-2a60f5c1ba6b" (UID: "2f369f02-bb75-4139-9eae-2a60f5c1ba6b"). InnerVolumeSpecName "kube-api-access-6fc5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:09:21.484130 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.484064    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-maas-api-tls" pod "2f369f02-bb75-4139-9eae-2a60f5c1ba6b" (UID: "2f369f02-bb75-4139-9eae-2a60f5c1ba6b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:09:21.583118 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.583083    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:21.583118 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.583117    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fc5x\" (UniqueName: \"kubernetes.io/projected/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-kube-api-access-6fc5x\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:21.583341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.583132    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:21.583341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.583147    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f369f02-bb75-4139-9eae-2a60f5c1ba6b-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:21.615089 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.615056    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:21.773253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.773160    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:21.819353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.819323    5066 generic.go:356] "Generic (PLEG): container finished" podID="2f369f02-bb75-4139-9eae-2a60f5c1ba6b" containerID="0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e" exitCode=0
Oct 30 06:09:21.819555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.819379    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798" event={"ID":"2f369f02-bb75-4139-9eae-2a60f5c1ba6b","Type":"ContainerDied","Data":"0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e"}
Oct 30 06:09:21.819555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.819383    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"
Oct 30 06:09:21.819555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.819423    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798" event={"ID":"2f369f02-bb75-4139-9eae-2a60f5c1ba6b","Type":"ContainerDied","Data":"d7d65185897de58005cd15d84200142acdc1db699081c79adadeee5bebf0f836"}
Oct 30 06:09:21.819555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.819445    5066 scope.go:122] "RemoveContainer" containerID="0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e"
Oct 30 06:09:21.831086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.831067    5066 scope.go:122] "RemoveContainer" containerID="0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e"
Oct 30 06:09:21.831385 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:09:21.831364    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e\": container with ID starting with 0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e not found: ID does not exist" containerID="0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e"
Oct 30 06:09:21.831439 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.831393    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e"} err="failed to get container status \"0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e\": rpc error: code = NotFound desc = could not find container \"0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e\": container with ID starting with 0197a518a6cc8c0e8b9b30b3d991b12ce684d3fdc954a6985cf80d9d92405b6e not found: ID does not exist"
Oct 30 06:09:21.881024 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.880986    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"]
Oct 30 06:09:21.893833 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:21.893806    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7ad9ad-d668b84d8-bc798"]
Oct 30 06:09:22.131264 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:22.131184    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:22.413665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:22.413634    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f369f02-bb75-4139-9eae-2a60f5c1ba6b" path="/var/lib/kubelet/pods/2f369f02-bb75-4139-9eae-2a60f5c1ba6b/volumes"
Oct 30 06:09:23.343465 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.343427    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"]
Oct 30 06:09:23.344349 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344325    5066 cpu_manager.go:414] "containerMap: removing container" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerName="istio-proxy"
Oct 30 06:09:23.344455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344349    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerName="istio-proxy"
Oct 30 06:09:23.344455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344369    5066 cpu_manager.go:414] "containerMap: removing container" podUID="7b475f41-1840-41fc-b58e-f1d623f8b6dd" containerName="maas-api"
Oct 30 06:09:23.344455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344376    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b475f41-1840-41fc-b58e-f1d623f8b6dd" containerName="maas-api"
Oct 30 06:09:23.344455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344409    5066 cpu_manager.go:414] "containerMap: removing container" podUID="9288193c-6696-4dd0-b4a9-4b2561949e06" containerName="istio-proxy"
Oct 30 06:09:23.344455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344416    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9288193c-6696-4dd0-b4a9-4b2561949e06" containerName="istio-proxy"
Oct 30 06:09:23.344455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344431    5066 cpu_manager.go:414] "containerMap: removing container" podUID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" containerName="istio-proxy"
Oct 30 06:09:23.344455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344439    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" containerName="istio-proxy"
Oct 30 06:09:23.344839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344474    5066 cpu_manager.go:414] "containerMap: removing container" podUID="2f369f02-bb75-4139-9eae-2a60f5c1ba6b" containerName="maas-api"
Oct 30 06:09:23.344839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344481    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f369f02-bb75-4139-9eae-2a60f5c1ba6b" containerName="maas-api"
Oct 30 06:09:23.344839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344494    5066 cpu_manager.go:414] "containerMap: removing container" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy"
Oct 30 06:09:23.344839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344501    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy"
Oct 30 06:09:23.344839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344689    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="51408510-bf0c-4ad2-b776-6dff1feb777b" containerName="istio-proxy"
Oct 30 06:09:23.344839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344707    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b475f41-1840-41fc-b58e-f1d623f8b6dd" containerName="maas-api"
Oct 30 06:09:23.344839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344722    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="9288193c-6696-4dd0-b4a9-4b2561949e06" containerName="istio-proxy"
Oct 30 06:09:23.344839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344734    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="0704d20f-f6e5-40b3-a61c-6afa67c7a8d6" containerName="istio-proxy"
Oct 30 06:09:23.344839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344748    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f369f02-bb75-4139-9eae-2a60f5c1ba6b" containerName="maas-api"
Oct 30 06:09:23.344839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.344758    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf2c8f49-dfe8-4b28-9634-c695bc74bf18" containerName="istio-proxy"
Oct 30 06:09:23.350161 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.350137    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.367021 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.366996    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-62316c-openshift-default-dockercfg-t276w\""
Oct 30 06:09:23.385053 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.385024    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"]
Oct 30 06:09:23.411944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.411905    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-socket\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.412133 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.412038    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-podinfo\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.412133 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.412085    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-credential-socket\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.412282 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.412250    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-envoy\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.412341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.412325    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5rcb\" (UniqueName: \"kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-kube-api-access-z5rcb\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.412414 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.412403    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-token\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.412560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.412532    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a325b74-cd9e-4678-a445-56cb549a12fb-istiod-ca-cert\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.412761 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.412734    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-data\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.412845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.412792    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-certs\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.438045 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.438013    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:23.514045 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514013    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a325b74-cd9e-4678-a445-56cb549a12fb-istiod-ca-cert\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.514230 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514071    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-data\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.514230 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514101    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-certs\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.514348 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514235    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-socket\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.514348 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514274    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-podinfo\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.514348 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514303    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-credential-socket\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.514523 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514389    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-envoy\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.514523 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514485    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5rcb\" (UniqueName: \"kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-kube-api-access-z5rcb\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.514663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514548    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-data\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.514663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514607    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-token\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.514663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514640    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-certs\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.514838 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514732    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-socket\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.514896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.514851    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a325b74-cd9e-4678-a445-56cb549a12fb-istiod-ca-cert\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.515116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.515090    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-credential-socket\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.516975 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.516955    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-envoy\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.517200 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.517182    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-podinfo\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.544645 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.544614    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5rcb\" (UniqueName: \"kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-kube-api-access-z5rcb\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.544800 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.544617    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-token\") pod \"e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") " pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:23.665210 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:23.665174    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:24.012163 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:09:24.012134    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a325b74_cd9e_4678_a445_56cb549a12fb.slice/crio-622320e37dc02aaf9c71945b8146e6fc76a05c406a7b65f2d7363145683dacca WatchSource:0}: Error finding container 622320e37dc02aaf9c71945b8146e6fc76a05c406a7b65f2d7363145683dacca: Status 404 returned error can't find the container with id 622320e37dc02aaf9c71945b8146e6fc76a05c406a7b65f2d7363145683dacca
Oct 30 06:09:24.013350 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:24.013316    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"]
Oct 30 06:09:24.015524 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:24.015482    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:09:24.015643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:24.015572    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:09:24.015643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:24.015631    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:09:24.856028 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:24.855991    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv" event={"ID":"1a325b74-cd9e-4678-a445-56cb549a12fb","Type":"ContainerStarted","Data":"b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe"}
Oct 30 06:09:24.856682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:24.856638    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv" event={"ID":"1a325b74-cd9e-4678-a445-56cb549a12fb","Type":"ContainerStarted","Data":"622320e37dc02aaf9c71945b8146e6fc76a05c406a7b65f2d7363145683dacca"}
Oct 30 06:09:24.953707 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:24.953640    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv" podStartSLOduration=1.953620876 podStartE2EDuration="1.953620876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:09:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:09:24.919774672 +0000 UTC m=+2268.665060007" watchObservedRunningTime="2026-10-30 06:09:24.953620876 +0000 UTC m=+2268.698906207"
Oct 30 06:09:25.131192 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:25.130854    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:25.314057 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:25.313945    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:25.481887 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:25.481854    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:25.665658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:25.665626    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:25.666845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:25.666820    5066 patch_prober.go:29] interesting pod/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Oct 30 06:09:25.666963 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:25.666889    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv" podUID="1a325b74-cd9e-4678-a445-56cb549a12fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Oct 30 06:09:26.665909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:26.665872    5066 patch_prober.go:29] interesting pod/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Oct 30 06:09:26.666313 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:26.665924    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv" podUID="1a325b74-cd9e-4678-a445-56cb549a12fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Oct 30 06:09:27.241613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:27.241309    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:27.685660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:27.683037    5066 patch_prober.go:29] interesting pod/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Oct 30 06:09:27.685660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:27.683109    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv" podUID="1a325b74-cd9e-4678-a445-56cb549a12fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Oct 30 06:09:28.450042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:28.450008    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:28.608332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:28.608305    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:28.665657 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:28.665621    5066 patch_prober.go:29] interesting pod/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Oct 30 06:09:28.665841 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:28.665683    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv" podUID="1a325b74-cd9e-4678-a445-56cb549a12fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Oct 30 06:09:28.903062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:28.902942    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:29.124391 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:29.124344    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:29.316758 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:29.316707    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:29.474058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:29.474003    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"]
Oct 30 06:09:29.475035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:29.474773    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl" podUID="5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" containerName="istio-proxy" containerID="cri-o://50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929" gracePeriod=30
Oct 30 06:09:29.633670 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:29.628677    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:29.674612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:29.669005    5066 patch_prober.go:29] interesting pod/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Oct 30 06:09:29.674612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:29.669077    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv" podUID="1a325b74-cd9e-4678-a445-56cb549a12fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Oct 30 06:09:30.376340 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:30.376299    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:30.683264 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:30.683194    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:30.824394 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:30.824323    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:30.824871 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:30.824841    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:30.825898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:30.825876    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:09:30.862540 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:30.862507    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:31.068774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:31.068683    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:31.400511 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:31.400425    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:31.904414 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:31.904364    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:32.485411 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:32.485365    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:32.588016 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:09:32.587977    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:09:32.810551 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:32.810459    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:34.209306 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.209265    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:34.490653 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.490617    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:34.671270 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.671231    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:34.774299 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.774270    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:09:34.818940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.818907    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-kube-api-access-g8b7g\" (UniqueName: \"kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-kube-api-access-g8b7g\") pod \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") "
Oct 30 06:09:34.818940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.818951    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-token\" (UniqueName: \"kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-token\") pod \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") "
Oct 30 06:09:34.819214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.819000    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istiod-ca-cert\") pod \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") "
Oct 30 06:09:34.819214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.819159    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-data\") pod \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") "
Oct 30 06:09:34.819214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.819200    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-envoy\") pod \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") "
Oct 30 06:09:34.819381 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.819234    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-credential-socket\") pod \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") "
Oct 30 06:09:34.819381 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.819356    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-socket\") pod \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") "
Oct 30 06:09:34.819494 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.819392    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-certs\") pod \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") "
Oct 30 06:09:34.819494 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.819439    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-podinfo\") pod \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\" (UID: \"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252\") "
Oct 30 06:09:34.819643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.819504    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-credential-socket" pod "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" (UID: "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:09:34.819643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.819577    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-socket" pod "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" (UID: "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:09:34.819767 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.819672    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istiod-ca-cert" pod "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" (UID: "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:09:34.819905 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.819883    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-data" pod "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" (UID: "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:09:34.820033 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.819907    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-certs" pod "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" (UID: "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:09:34.820442 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.820422    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:34.820537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.820448    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:34.820537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.820463    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:34.820537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.820476    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:34.820537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.820489    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:34.821570 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.821549    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-token" pod "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" (UID: "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:09:34.822077 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.822049    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-envoy" pod "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" (UID: "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:09:34.822164 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.822095    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-kube-api-access-g8b7g" pod "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" (UID: "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252"). InnerVolumeSpecName "kube-api-access-g8b7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:09:34.822164 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.822135    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-podinfo" pod "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" (UID: "5d0d0d4d-3d1b-4be6-b9a0-2367550b0252"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:09:34.921702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.921668    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:34.921702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.921698    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:34.921702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.921708    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8b7g\" (UniqueName: \"kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-kube-api-access-g8b7g\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:34.922007 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:34.921717    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:09:35.054625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:35.054512    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:35.187038 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:35.187008    5066 generic.go:356] "Generic (PLEG): container finished" podID="5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" containerID="50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929" exitCode=0
Oct 30 06:09:35.187255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:35.187056    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl" event={"ID":"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252","Type":"ContainerDied","Data":"50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929"}
Oct 30 06:09:35.187255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:35.187098    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl" event={"ID":"5d0d0d4d-3d1b-4be6-b9a0-2367550b0252","Type":"ContainerDied","Data":"676c2bd144c95dd9aa5708cee9466fb35cab0a756ed377cdfe9c2355d46d56ef"}
Oct 30 06:09:35.187255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:35.187129    5066 scope.go:122] "RemoveContainer" containerID="50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929"
Oct 30 06:09:35.187255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:35.187061    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"
Oct 30 06:09:35.199012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:35.198992    5066 scope.go:122] "RemoveContainer" containerID="50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929"
Oct 30 06:09:35.199318 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:09:35.199299    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929\": container with ID starting with 50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929 not found: ID does not exist" containerID="50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929"
Oct 30 06:09:35.199367 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:35.199330    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929"} err="failed to get container status \"50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929\": rpc error: code = NotFound desc = could not find container \"50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929\": container with ID starting with 50d55b63eefe614618624800070340e48fb91ff24f39051567db9e6631ce8929 not found: ID does not exist"
Oct 30 06:09:35.235869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:35.235836    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:35.262155 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:35.262125    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"]
Oct 30 06:09:35.275860 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:35.275830    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-7ad9ad-openshift-default-78bdfc7bd9-r2gjl"]
Oct 30 06:09:36.019961 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:36.019914    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:36.407331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:36.407302    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" path="/var/lib/kubelet/pods/5d0d0d4d-3d1b-4be6-b9a0-2367550b0252/volumes"
Oct 30 06:09:36.650699 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:36.650670    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:36.825759 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:36.825664    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:36.981992 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:36.981951    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"]
Oct 30 06:09:36.983393 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:36.983366    5066 cpu_manager.go:414] "containerMap: removing container" podUID="5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" containerName="istio-proxy"
Oct 30 06:09:36.983557 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:36.983546    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" containerName="istio-proxy"
Oct 30 06:09:36.983888 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:36.983874    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d0d0d4d-3d1b-4be6-b9a0-2367550b0252" containerName="istio-proxy"
Oct 30 06:09:36.990342 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:36.990318    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:36.996805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:36.996773    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"]
Oct 30 06:09:37.018571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.007831    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w4-815776\"/\"sub-test-model-815776-kserve-self-signed-certs\""
Oct 30 06:09:37.018571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.007945    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w4-815776\"/\"default-dockercfg-mlrkr\""
Oct 30 06:09:37.018571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.008009    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w4-815776\"/\"kube-root-ca.crt\""
Oct 30 06:09:37.022236 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.021517    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w4-815776\"/\"openshift-service-ca.crt\""
Oct 30 06:09:37.082825 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.082725    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-model-cache\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.082989 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.082908    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tls-certs\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.082989 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.082949    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-dshm\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.082989 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.082973    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-home\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.083159 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.082998    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tmp-dir\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.083159 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.083038    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvdl2\" (UniqueName: \"kubernetes.io/projected/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-kube-api-access-bvdl2\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.111295 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.111232    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:37.184532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.184504    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-model-cache\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.184691 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.184642    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tls-certs\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.184832 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.184813    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-dshm\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.184906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.184845    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-home\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.184906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.184876    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tmp-dir\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.185046 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.184916    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvdl2\" (UniqueName: \"kubernetes.io/projected/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-kube-api-access-bvdl2\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.185046 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.184924    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-model-cache\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.185925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.185902    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-home\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.186042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.185931    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tmp-dir\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.188718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.188695    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tls-certs\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.192238 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.192211    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-dshm\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.214256 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.214219    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvdl2\" (UniqueName: \"kubernetes.io/projected/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-kube-api-access-bvdl2\") pod \"sub-test-model-815776-kserve-59bff6cd69-7skwp\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") " pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.344103 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.342935    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:37.725304 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.725270    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"]
Oct 30 06:09:37.726376 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:09:37.726340    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1744a5e_e05e_4b57_aa48_3bfa02672ee4.slice/crio-f4ee10e8e4651b1d9887f30a34ae1f7c5ab453e3fe25de9450cf2e203d3ab884 WatchSource:0}: Error finding container f4ee10e8e4651b1d9887f30a34ae1f7c5ab453e3fe25de9450cf2e203d3ab884: Status 404 returned error can't find the container with id f4ee10e8e4651b1d9887f30a34ae1f7c5ab453e3fe25de9450cf2e203d3ab884
Oct 30 06:09:37.852810 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:37.852779    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:38.217338 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:38.217305    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp" event={"ID":"c1744a5e-e05e-4b57-aa48-3bfa02672ee4","Type":"ContainerStarted","Data":"531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802"}
Oct 30 06:09:38.217532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:38.217349    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp" event={"ID":"c1744a5e-e05e-4b57-aa48-3bfa02672ee4","Type":"ContainerStarted","Data":"f4ee10e8e4651b1d9887f30a34ae1f7c5ab453e3fe25de9450cf2e203d3ab884"}
Oct 30 06:09:38.267003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:38.266957    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp" podStartSLOduration=2.266937085 podStartE2EDuration="2.266937085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:09:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:09:38.262799125 +0000 UTC m=+2282.008084460" watchObservedRunningTime="2026-10-30 06:09:38.266937085 +0000 UTC m=+2282.012222417"
Oct 30 06:09:38.343996 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:38.343964    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:39.575397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:39.572961    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:41.179774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:41.179739    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:42.653788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:42.653753    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:44.030256 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:44.030189    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:44.329224 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:44.329125    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:45.865634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:45.865578    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:47.343448 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:47.343366    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:47.357689 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:47.357658    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:47.687187 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:47.687087    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:48.300676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:48.300649    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:09:48.936740 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:48.936704    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:49.093681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:49.093655    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:49.346223 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:49.346098    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:50.671491 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:50.671445    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:51.164211 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:51.164180    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:51.330613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:51.330055    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:51.494859 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:51.494772    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:51.786090 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:51.785998    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:53.192985 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:53.192922    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:54.944639 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:54.944601    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:56.431656 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:56.431581    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:57.994172 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:57.994133    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:59.198077 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:59.198035    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:59.388807 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:59.388772    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:59.467336 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:59.467238    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"]
Oct 30 06:09:59.467786 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:59.467755    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" podUID="40e57278-1ad4-4d16-b8be-fa1f992467ae" containerName="istio-proxy" containerID="cri-o://75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86" gracePeriod=30
Oct 30 06:09:59.583391 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:59.583356    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:09:59.864334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:09:59.864238    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:00.456788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:00.456729    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:01.047695 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:01.047662    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:01.245018 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:01.244985    5066 patch_prober.go:29] interesting pod/e2e-inf-b-c54211-openshift-default-767566b477-blgbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:10:01.245375 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:01.245345    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" podUID="40e57278-1ad4-4d16-b8be-fa1f992467ae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:10:01.373607 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:01.373499    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:01.854499 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:01.854431    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:02.554346 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:10:02.554315    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:10:03.363621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:03.362456    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:04.799861 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.799835    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:10:04.907055 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.907024    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-data\") pod \"40e57278-1ad4-4d16-b8be-fa1f992467ae\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") "
Oct 30 06:10:04.907255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.907139    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-kube-api-access-p6vz9\" (UniqueName: \"kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-kube-api-access-p6vz9\") pod \"40e57278-1ad4-4d16-b8be-fa1f992467ae\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") "
Oct 30 06:10:04.907255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.907204    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-podinfo\") pod \"40e57278-1ad4-4d16-b8be-fa1f992467ae\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") "
Oct 30 06:10:04.907255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.907253    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-token\" (UniqueName: \"kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-token\") pod \"40e57278-1ad4-4d16-b8be-fa1f992467ae\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") "
Oct 30 06:10:04.907430 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.907321    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-socket\") pod \"40e57278-1ad4-4d16-b8be-fa1f992467ae\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") "
Oct 30 06:10:04.907430 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.907354    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40e57278-1ad4-4d16-b8be-fa1f992467ae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40e57278-1ad4-4d16-b8be-fa1f992467ae-istiod-ca-cert\") pod \"40e57278-1ad4-4d16-b8be-fa1f992467ae\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") "
Oct 30 06:10:04.907430 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.907401    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-envoy\") pod \"40e57278-1ad4-4d16-b8be-fa1f992467ae\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") "
Oct 30 06:10:04.907602 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.907461    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-credential-socket\") pod \"40e57278-1ad4-4d16-b8be-fa1f992467ae\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") "
Oct 30 06:10:04.907602 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.907528    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-certs\") pod \"40e57278-1ad4-4d16-b8be-fa1f992467ae\" (UID: \"40e57278-1ad4-4d16-b8be-fa1f992467ae\") "
Oct 30 06:10:04.908752 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.908721    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-data" pod "40e57278-1ad4-4d16-b8be-fa1f992467ae" (UID: "40e57278-1ad4-4d16-b8be-fa1f992467ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:04.909167 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.909006    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40e57278-1ad4-4d16-b8be-fa1f992467ae-istiod-ca-cert" pod "40e57278-1ad4-4d16-b8be-fa1f992467ae" (UID: "40e57278-1ad4-4d16-b8be-fa1f992467ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:10:04.909167 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.909099    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-credential-socket" pod "40e57278-1ad4-4d16-b8be-fa1f992467ae" (UID: "40e57278-1ad4-4d16-b8be-fa1f992467ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:04.909770 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.909735    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-socket" pod "40e57278-1ad4-4d16-b8be-fa1f992467ae" (UID: "40e57278-1ad4-4d16-b8be-fa1f992467ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:04.909995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.909975    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-certs" pod "40e57278-1ad4-4d16-b8be-fa1f992467ae" (UID: "40e57278-1ad4-4d16-b8be-fa1f992467ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:04.911690 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.911662    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-podinfo" pod "40e57278-1ad4-4d16-b8be-fa1f992467ae" (UID: "40e57278-1ad4-4d16-b8be-fa1f992467ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:10:04.912517 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.912492    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-token" pod "40e57278-1ad4-4d16-b8be-fa1f992467ae" (UID: "40e57278-1ad4-4d16-b8be-fa1f992467ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:04.912964 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.912934    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-kube-api-access-p6vz9" pod "40e57278-1ad4-4d16-b8be-fa1f992467ae" (UID: "40e57278-1ad4-4d16-b8be-fa1f992467ae"). InnerVolumeSpecName "kube-api-access-p6vz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:04.913091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.913050    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-envoy" pod "40e57278-1ad4-4d16-b8be-fa1f992467ae" (UID: "40e57278-1ad4-4d16-b8be-fa1f992467ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:04.958494 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:04.958453    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:05.009555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.009520    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:05.009555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.009545    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6vz9\" (UniqueName: \"kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-kube-api-access-p6vz9\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:05.009555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.009557    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:05.009555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.009566    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:05.009871 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.009575    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:05.009871 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.009623    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40e57278-1ad4-4d16-b8be-fa1f992467ae-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:05.009871 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.009639    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:05.009871 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.009650    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:05.009871 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.009659    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40e57278-1ad4-4d16-b8be-fa1f992467ae-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:05.136576 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.136361    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:05.444439 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.444411    5066 generic.go:356] "Generic (PLEG): container finished" podID="40e57278-1ad4-4d16-b8be-fa1f992467ae" containerID="75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86" exitCode=0
Oct 30 06:10:05.444649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.444463    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"
Oct 30 06:10:05.444649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.444491    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" event={"ID":"40e57278-1ad4-4d16-b8be-fa1f992467ae","Type":"ContainerDied","Data":"75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86"}
Oct 30 06:10:05.444649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.444548    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh" event={"ID":"40e57278-1ad4-4d16-b8be-fa1f992467ae","Type":"ContainerDied","Data":"99373b5f561bb25e93163ab07c963b6d135b9aa92dea0520f63dfb5e9a49a9e6"}
Oct 30 06:10:05.444649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.444572    5066 scope.go:122] "RemoveContainer" containerID="75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86"
Oct 30 06:10:05.457429 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.457410    5066 scope.go:122] "RemoveContainer" containerID="75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86"
Oct 30 06:10:05.460637 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:10:05.460529    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86\": container with ID starting with 75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86 not found: ID does not exist" containerID="75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86"
Oct 30 06:10:05.460637 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.460567    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86"} err="failed to get container status \"75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86\": rpc error: code = NotFound desc = could not find container \"75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86\": container with ID starting with 75a08bf080bb0bd2bad6f053fb5d01aa6e3c7c2babcace0def863a3e94d6ce86 not found: ID does not exist"
Oct 30 06:10:05.504923 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.504893    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"]
Oct 30 06:10:05.519157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:05.519128    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c54211-openshift-default-767566b477-blgbh"]
Oct 30 06:10:06.405355 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:06.405323    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40e57278-1ad4-4d16-b8be-fa1f992467ae" path="/var/lib/kubelet/pods/40e57278-1ad4-4d16-b8be-fa1f992467ae/volumes"
Oct 30 06:10:06.559497 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:06.559444    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:06.729389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:06.729262    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:06.916657 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:06.915784    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:08.478034 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:08.477988    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:10.138296 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:10.138266    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:10.821368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:10.821333    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:12.189234 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:12.189200    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:12.362357 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:12.362325    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:14.126476 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:14.126436    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:15.178456 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:15.178421    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:15.835737 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:15.835706    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:17.566201 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:17.566165    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:19.016259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:19.016222    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:20.801874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:20.801838    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:21.462739 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:21.462708    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:21.499486 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:21.499450    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"]
Oct 30 06:10:21.499722 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:21.499696    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy" containerID="cri-o://eb9dfff5edd2f1dadc273e34f832919c628c0a989ac53d263f6025cc93ee78c8" gracePeriod=30
Oct 30 06:10:21.515181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:21.515147    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"]
Oct 30 06:10:21.515404 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:21.515371    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerName="istio-proxy" containerID="cri-o://1d92f01ed1cac08614493d8af497d915f93a845f76bee51ef4830dfc2e59ac7e" gracePeriod=30
Oct 30 06:10:21.518688 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:21.518666    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"]
Oct 30 06:10:21.518872 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:21.518850    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy" containerID="cri-o://ce4307ba80c4bf2564ac9ae5f16d90d905b5085383f2092bff694220d5e6186d" gracePeriod=30
Oct 30 06:10:21.623711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:21.623677    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:22.517840 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:22.517803    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:23.927771 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:23.927736    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:24.444112 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:24.444062    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:24.608450 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:24.608409    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:24.774940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:24.774853    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:24.864437 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:24.864374    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"]
Oct 30 06:10:24.865062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:24.865029    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy" containerID="cri-o://7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5" gracePeriod=30
Oct 30 06:10:25.103838 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:25.103748    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:25.313779 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:25.313748    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xfps7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:10:25.313976 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:25.313817    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:10:25.577349 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:25.577310    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-p99cc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:10:25.577537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:25.577418    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:10:25.621250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:25.621204    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-xjhtt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:10:25.621426 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:25.621273    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:10:25.858683 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:25.858559    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:26.021418 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:26.021392    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:26.602010 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:26.601979    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:26.674023 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:26.673986    5066 generic.go:356] "Generic (PLEG): container finished" podID="204b23d9-e64b-475c-be79-4a3746371200" containerID="eb9dfff5edd2f1dadc273e34f832919c628c0a989ac53d263f6025cc93ee78c8" exitCode=0
Oct 30 06:10:26.674224 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:26.674032    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" event={"ID":"204b23d9-e64b-475c-be79-4a3746371200","Type":"ContainerDied","Data":"eb9dfff5edd2f1dadc273e34f832919c628c0a989ac53d263f6025cc93ee78c8"}
Oct 30 06:10:26.676496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:26.676467    5066 generic.go:356] "Generic (PLEG): container finished" podID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerID="ce4307ba80c4bf2564ac9ae5f16d90d905b5085383f2092bff694220d5e6186d" exitCode=0
Oct 30 06:10:26.676708 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:26.676561    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" event={"ID":"cd59ca3c-8f24-4f58-bd47-01e6eae62873","Type":"ContainerDied","Data":"ce4307ba80c4bf2564ac9ae5f16d90d905b5085383f2092bff694220d5e6186d"}
Oct 30 06:10:26.678875 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:26.678846    5066 generic.go:356] "Generic (PLEG): container finished" podID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerID="1d92f01ed1cac08614493d8af497d915f93a845f76bee51ef4830dfc2e59ac7e" exitCode=0
Oct 30 06:10:26.679064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:26.678959    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" event={"ID":"2736e286-37f0-4c52-9e22-eab3c3b52b5e","Type":"ContainerDied","Data":"1d92f01ed1cac08614493d8af497d915f93a845f76bee51ef4830dfc2e59ac7e"}
Oct 30 06:10:26.877851 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:26.870281    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 06:10:26.921022 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:26.920996    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 06:10:26.956836 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:26.956806    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 06:10:27.033295 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033259    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-credential-socket\") pod \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") "
Oct 30 06:10:27.033295 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033300    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/204b23d9-e64b-475c-be79-4a3746371200-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/204b23d9-e64b-475c-be79-4a3746371200-istiod-ca-cert\") pod \"204b23d9-e64b-475c-be79-4a3746371200\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") "
Oct 30 06:10:27.033541 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033342    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-credential-socket\") pod \"204b23d9-e64b-475c-be79-4a3746371200\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") "
Oct 30 06:10:27.033541 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033367    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/204b23d9-e64b-475c-be79-4a3746371200-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/204b23d9-e64b-475c-be79-4a3746371200-istio-podinfo\") pod \"204b23d9-e64b-475c-be79-4a3746371200\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") "
Oct 30 06:10:27.033541 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033439    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-envoy\") pod \"204b23d9-e64b-475c-be79-4a3746371200\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") "
Oct 30 06:10:27.033541 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033465    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-data\") pod \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") "
Oct 30 06:10:27.033541 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033526    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-certs\") pod \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") "
Oct 30 06:10:27.033805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033561    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-envoy\") pod \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") "
Oct 30 06:10:27.033805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033649    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-credential-socket" pod "cd59ca3c-8f24-4f58-bd47-01e6eae62873" (UID: "cd59ca3c-8f24-4f58-bd47-01e6eae62873"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.033805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033682    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-certs\") pod \"204b23d9-e64b-475c-be79-4a3746371200\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") "
Oct 30 06:10:27.033805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033745    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istiod-ca-cert\") pod \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") "
Oct 30 06:10:27.033805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033752    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-credential-socket" pod "204b23d9-e64b-475c-be79-4a3746371200" (UID: "204b23d9-e64b-475c-be79-4a3746371200"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.034064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033872    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-kube-api-access-nv9fc\" (UniqueName: \"kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-kube-api-access-nv9fc\") pod \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") "
Oct 30 06:10:27.034064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033886    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-certs" pod "204b23d9-e64b-475c-be79-4a3746371200" (UID: "204b23d9-e64b-475c-be79-4a3746371200"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.034064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033926    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-certs" pod "cd59ca3c-8f24-4f58-bd47-01e6eae62873" (UID: "cd59ca3c-8f24-4f58-bd47-01e6eae62873"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.034064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.033935    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/204b23d9-e64b-475c-be79-4a3746371200-istiod-ca-cert" pod "204b23d9-e64b-475c-be79-4a3746371200" (UID: "204b23d9-e64b-475c-be79-4a3746371200"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:10:27.034321 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034098    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-token\" (UniqueName: \"kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-token\") pod \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") "
Oct 30 06:10:27.034321 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034107    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-data" pod "cd59ca3c-8f24-4f58-bd47-01e6eae62873" (UID: "cd59ca3c-8f24-4f58-bd47-01e6eae62873"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.034321 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034149    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-token\" (UniqueName: \"kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-token\") pod \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") "
Oct 30 06:10:27.034321 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034174    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istiod-ca-cert" pod "cd59ca3c-8f24-4f58-bd47-01e6eae62873" (UID: "cd59ca3c-8f24-4f58-bd47-01e6eae62873"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:10:27.034321 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034189    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-kube-api-access-7kbbb\" (UniqueName: \"kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-kube-api-access-7kbbb\") pod \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") "
Oct 30 06:10:27.034321 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034248    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-podinfo\") pod \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") "
Oct 30 06:10:27.034321 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034282    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-kube-api-access-lfp5x\" (UniqueName: \"kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-kube-api-access-lfp5x\") pod \"204b23d9-e64b-475c-be79-4a3746371200\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") "
Oct 30 06:10:27.034947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034324    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-istio-token\" (UniqueName: \"kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-istio-token\") pod \"204b23d9-e64b-475c-be79-4a3746371200\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") "
Oct 30 06:10:27.034947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034369    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-envoy\") pod \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") "
Oct 30 06:10:27.034947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034399    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-podinfo\") pod \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") "
Oct 30 06:10:27.034947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034438    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-socket\") pod \"204b23d9-e64b-475c-be79-4a3746371200\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") "
Oct 30 06:10:27.034947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034484    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-socket\") pod \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\" (UID: \"cd59ca3c-8f24-4f58-bd47-01e6eae62873\") "
Oct 30 06:10:27.034947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.034532    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-data\") pod \"204b23d9-e64b-475c-be79-4a3746371200\" (UID: \"204b23d9-e64b-475c-be79-4a3746371200\") "
Oct 30 06:10:27.037717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.036419    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.037717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.036445    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/204b23d9-e64b-475c-be79-4a3746371200-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.037717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.036458    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.037717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.036471    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.037717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.036483    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.037717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.036494    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.037717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.036505    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.037717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.037166    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-envoy" pod "cd59ca3c-8f24-4f58-bd47-01e6eae62873" (UID: "cd59ca3c-8f24-4f58-bd47-01e6eae62873"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.037717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.037373    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-token" pod "cd59ca3c-8f24-4f58-bd47-01e6eae62873" (UID: "cd59ca3c-8f24-4f58-bd47-01e6eae62873"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:27.038313 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.037876    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-kube-api-access-7kbbb" pod "2736e286-37f0-4c52-9e22-eab3c3b52b5e" (UID: "2736e286-37f0-4c52-9e22-eab3c3b52b5e"). InnerVolumeSpecName "kube-api-access-7kbbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:27.038313 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.038166    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-token" pod "2736e286-37f0-4c52-9e22-eab3c3b52b5e" (UID: "2736e286-37f0-4c52-9e22-eab3c3b52b5e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:27.038668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.038645    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-socket" pod "cd59ca3c-8f24-4f58-bd47-01e6eae62873" (UID: "cd59ca3c-8f24-4f58-bd47-01e6eae62873"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.038911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.038884    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-socket" pod "204b23d9-e64b-475c-be79-4a3746371200" (UID: "204b23d9-e64b-475c-be79-4a3746371200"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.039408 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.039379    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/204b23d9-e64b-475c-be79-4a3746371200-istio-podinfo" pod "204b23d9-e64b-475c-be79-4a3746371200" (UID: "204b23d9-e64b-475c-be79-4a3746371200"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:10:27.039918 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.039884    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-kube-api-access-nv9fc" pod "cd59ca3c-8f24-4f58-bd47-01e6eae62873" (UID: "cd59ca3c-8f24-4f58-bd47-01e6eae62873"). InnerVolumeSpecName "kube-api-access-nv9fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:27.040276 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.040248    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-data" pod "204b23d9-e64b-475c-be79-4a3746371200" (UID: "204b23d9-e64b-475c-be79-4a3746371200"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.051881 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.051696    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-podinfo" pod "cd59ca3c-8f24-4f58-bd47-01e6eae62873" (UID: "cd59ca3c-8f24-4f58-bd47-01e6eae62873"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:10:27.051881 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.051765    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-istio-token" pod "204b23d9-e64b-475c-be79-4a3746371200" (UID: "204b23d9-e64b-475c-be79-4a3746371200"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:27.051881 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.051791    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-envoy" pod "204b23d9-e64b-475c-be79-4a3746371200" (UID: "204b23d9-e64b-475c-be79-4a3746371200"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.051881 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.051839    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-kube-api-access-lfp5x" pod "204b23d9-e64b-475c-be79-4a3746371200" (UID: "204b23d9-e64b-475c-be79-4a3746371200"). InnerVolumeSpecName "kube-api-access-lfp5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:27.051881 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.051850    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-envoy" pod "2736e286-37f0-4c52-9e22-eab3c3b52b5e" (UID: "2736e286-37f0-4c52-9e22-eab3c3b52b5e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.051881 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.051860    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-podinfo" pod "2736e286-37f0-4c52-9e22-eab3c3b52b5e" (UID: "2736e286-37f0-4c52-9e22-eab3c3b52b5e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:10:27.138106 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.138026    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istiod-ca-cert\") pod \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") "
Oct 30 06:10:27.138106 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.138073    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-socket\") pod \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") "
Oct 30 06:10:27.138106 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.138105    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-data\") pod \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") "
Oct 30 06:10:27.138382 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.138176    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-credential-socket\") pod \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") "
Oct 30 06:10:27.138382 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.138297    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-certs\") pod \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\" (UID: \"2736e286-37f0-4c52-9e22-eab3c3b52b5e\") "
Oct 30 06:10:27.138490 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.138392    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-socket" pod "2736e286-37f0-4c52-9e22-eab3c3b52b5e" (UID: "2736e286-37f0-4c52-9e22-eab3c3b52b5e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.138490 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.138452    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istiod-ca-cert" pod "2736e286-37f0-4c52-9e22-eab3c3b52b5e" (UID: "2736e286-37f0-4c52-9e22-eab3c3b52b5e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:10:27.138619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.138571    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-credential-socket" pod "2736e286-37f0-4c52-9e22-eab3c3b52b5e" (UID: "2736e286-37f0-4c52-9e22-eab3c3b52b5e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.138683 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.138663    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-data" pod "2736e286-37f0-4c52-9e22-eab3c3b52b5e" (UID: "2736e286-37f0-4c52-9e22-eab3c3b52b5e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.138743 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.138683    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-certs" pod "2736e286-37f0-4c52-9e22-eab3c3b52b5e" (UID: "2736e286-37f0-4c52-9e22-eab3c3b52b5e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:27.139034 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139015    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/204b23d9-e64b-475c-be79-4a3746371200-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139110 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139038    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139110 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139051    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139110 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139063    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139110 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139075    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139110 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139088    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv9fc\" (UniqueName: \"kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-kube-api-access-nv9fc\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139110 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139102    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139115    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139128    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kbbb\" (UniqueName: \"kubernetes.io/projected/2736e286-37f0-4c52-9e22-eab3c3b52b5e-kube-api-access-7kbbb\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139171    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd59ca3c-8f24-4f58-bd47-01e6eae62873-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139184    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfp5x\" (UniqueName: \"kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-kube-api-access-lfp5x\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139198    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/204b23d9-e64b-475c-be79-4a3746371200-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139232    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139246    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139259    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139272    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd59ca3c-8f24-4f58-bd47-01e6eae62873-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139285    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/204b23d9-e64b-475c-be79-4a3746371200-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139298    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139310    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.139389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.139322    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2736e286-37f0-4c52-9e22-eab3c3b52b5e-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:27.687283 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.687255    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"
Oct 30 06:10:27.687801 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.687282    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7" event={"ID":"2736e286-37f0-4c52-9e22-eab3c3b52b5e","Type":"ContainerDied","Data":"15b53e84d6a1da8cd44f923b57bf8d9c88c6f0d326d80a1b463aabdee8be0dee"}
Oct 30 06:10:27.687801 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.687346    5066 scope.go:122] "RemoveContainer" containerID="1d92f01ed1cac08614493d8af497d915f93a845f76bee51ef4830dfc2e59ac7e"
Oct 30 06:10:27.689175 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.689148    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc" event={"ID":"204b23d9-e64b-475c-be79-4a3746371200","Type":"ContainerDied","Data":"c8e31ec710cf286ddc0e01307632da18043c2bef585a0bc0a683211ccb397b74"}
Oct 30 06:10:27.689277 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.689263    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"
Oct 30 06:10:27.697263 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.697236    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt" event={"ID":"cd59ca3c-8f24-4f58-bd47-01e6eae62873","Type":"ContainerDied","Data":"7c519783d975324f35a2bbd7be9dbc8988e2eb59070db03d8d4502bfebb17e08"}
Oct 30 06:10:27.697263 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.697264    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"
Oct 30 06:10:27.699038 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.698843    5066 scope.go:122] "RemoveContainer" containerID="eb9dfff5edd2f1dadc273e34f832919c628c0a989ac53d263f6025cc93ee78c8"
Oct 30 06:10:27.710183 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.710159    5066 scope.go:122] "RemoveContainer" containerID="ce4307ba80c4bf2564ac9ae5f16d90d905b5085383f2092bff694220d5e6186d"
Oct 30 06:10:27.747251 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.747221    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"]
Oct 30 06:10:27.761009 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.760976    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-p99cc"]
Oct 30 06:10:27.791403 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.791308    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"]
Oct 30 06:10:27.797627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.797582    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xfps7"]
Oct 30 06:10:27.852302 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.852269    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"]
Oct 30 06:10:27.855542 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:27.855519    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-xjhtt"]
Oct 30 06:10:28.288094 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:28.288061    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:28.408047 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:28.408016    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="204b23d9-e64b-475c-be79-4a3746371200" path="/var/lib/kubelet/pods/204b23d9-e64b-475c-be79-4a3746371200/volumes"
Oct 30 06:10:28.408541 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:28.408527    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" path="/var/lib/kubelet/pods/2736e286-37f0-4c52-9e22-eab3c3b52b5e/volumes"
Oct 30 06:10:28.408994 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:28.408980    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" path="/var/lib/kubelet/pods/cd59ca3c-8f24-4f58-bd47-01e6eae62873/volumes"
Oct 30 06:10:29.411895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:29.411856    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:10:29.412307 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:29.411915    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:10:29.927972 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:29.927941    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:30.164329 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.164297    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 06:10:30.181702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.181668    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-kube-api-access-8jtcb\" (UniqueName: \"kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-kube-api-access-8jtcb\") pod \"8f65f330-0c39-4391-8de1-8d793b792708\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") "
Oct 30 06:10:30.181863 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.181777    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-credential-socket\") pod \"8f65f330-0c39-4391-8de1-8d793b792708\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") "
Oct 30 06:10:30.181938 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.181896    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-envoy\") pod \"8f65f330-0c39-4391-8de1-8d793b792708\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") "
Oct 30 06:10:30.182000 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.181953    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-data\") pod \"8f65f330-0c39-4391-8de1-8d793b792708\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") "
Oct 30 06:10:30.182060 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.182007    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8f65f330-0c39-4391-8de1-8d793b792708-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f65f330-0c39-4391-8de1-8d793b792708-istio-podinfo\") pod \"8f65f330-0c39-4391-8de1-8d793b792708\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") "
Oct 30 06:10:30.182128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.182112    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-istio-token\" (UniqueName: \"kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-istio-token\") pod \"8f65f330-0c39-4391-8de1-8d793b792708\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") "
Oct 30 06:10:30.182252 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.182234    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8f65f330-0c39-4391-8de1-8d793b792708-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f65f330-0c39-4391-8de1-8d793b792708-istiod-ca-cert\") pod \"8f65f330-0c39-4391-8de1-8d793b792708\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") "
Oct 30 06:10:30.182443 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.182412    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-credential-socket" pod "8f65f330-0c39-4391-8de1-8d793b792708" (UID: "8f65f330-0c39-4391-8de1-8d793b792708"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:30.182628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.182600    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-data" pod "8f65f330-0c39-4391-8de1-8d793b792708" (UID: "8f65f330-0c39-4391-8de1-8d793b792708"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:30.183008 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.182980    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:30.183142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.183012    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:30.183480 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.183454    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f65f330-0c39-4391-8de1-8d793b792708-istiod-ca-cert" pod "8f65f330-0c39-4391-8de1-8d793b792708" (UID: "8f65f330-0c39-4391-8de1-8d793b792708"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:10:30.185677 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.185650    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-envoy" pod "8f65f330-0c39-4391-8de1-8d793b792708" (UID: "8f65f330-0c39-4391-8de1-8d793b792708"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:30.190022 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.190000    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f65f330-0c39-4391-8de1-8d793b792708-istio-podinfo" pod "8f65f330-0c39-4391-8de1-8d793b792708" (UID: "8f65f330-0c39-4391-8de1-8d793b792708"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:10:30.190153 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.190044    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-istio-token" pod "8f65f330-0c39-4391-8de1-8d793b792708" (UID: "8f65f330-0c39-4391-8de1-8d793b792708"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:30.190153 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.190067    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-kube-api-access-8jtcb" pod "8f65f330-0c39-4391-8de1-8d793b792708" (UID: "8f65f330-0c39-4391-8de1-8d793b792708"). InnerVolumeSpecName "kube-api-access-8jtcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:30.284510 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.284475    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-socket\") pod \"8f65f330-0c39-4391-8de1-8d793b792708\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") "
Oct 30 06:10:30.284706 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.284552    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-certs\") pod \"8f65f330-0c39-4391-8de1-8d793b792708\" (UID: \"8f65f330-0c39-4391-8de1-8d793b792708\") "
Oct 30 06:10:30.284858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.284772    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-socket" pod "8f65f330-0c39-4391-8de1-8d793b792708" (UID: "8f65f330-0c39-4391-8de1-8d793b792708"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:30.284910 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.284877    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-certs" pod "8f65f330-0c39-4391-8de1-8d793b792708" (UID: "8f65f330-0c39-4391-8de1-8d793b792708"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:30.285215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.285199    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:30.285257 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.285217    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f65f330-0c39-4391-8de1-8d793b792708-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:30.285257 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.285226    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:30.285257 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.285236    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:30.285257 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.285244    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f65f330-0c39-4391-8de1-8d793b792708-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:30.285257 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.285252    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f65f330-0c39-4391-8de1-8d793b792708-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:30.285420 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.285260    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jtcb\" (UniqueName: \"kubernetes.io/projected/8f65f330-0c39-4391-8de1-8d793b792708-kube-api-access-8jtcb\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:30.731359 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.731275    5066 generic.go:356] "Generic (PLEG): container finished" podID="8f65f330-0c39-4391-8de1-8d793b792708" containerID="7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5" exitCode=0
Oct 30 06:10:30.731359 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.731329    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"
Oct 30 06:10:30.731842 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.731359    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" event={"ID":"8f65f330-0c39-4391-8de1-8d793b792708","Type":"ContainerDied","Data":"7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5"}
Oct 30 06:10:30.731842 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.731413    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz" event={"ID":"8f65f330-0c39-4391-8de1-8d793b792708","Type":"ContainerDied","Data":"3975e12f8a6e63f7db04339546b6cc4657ac2cbb27dc95e5b9d2ca0a3d1de101"}
Oct 30 06:10:30.731842 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.731438    5066 scope.go:122] "RemoveContainer" containerID="7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5"
Oct 30 06:10:30.741767 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.741750    5066 scope.go:122] "RemoveContainer" containerID="7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5"
Oct 30 06:10:30.742054 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:10:30.742032    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5\": container with ID starting with 7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5 not found: ID does not exist" containerID="7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5"
Oct 30 06:10:30.742115 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.742064    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5"} err="failed to get container status \"7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5\": rpc error: code = NotFound desc = could not find container \"7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5\": container with ID starting with 7d14b189b3ea9ad31341fb34276e8213216f85a6c329686ce5dcc1c048cfc7e5 not found: ID does not exist"
Oct 30 06:10:30.790058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.790030    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"]
Oct 30 06:10:30.804556 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:30.804528    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-8jvdz"]
Oct 30 06:10:31.500846 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:31.500812    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:31.795672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:31.795549    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:31.965771 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:31.965724    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:32.117497 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:32.117398    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"]
Oct 30 06:10:32.117709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:32.117675    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" podUID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" containerName="istio-proxy" containerID="cri-o://62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462" gracePeriod=30
Oct 30 06:10:32.142528 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:32.142496    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:32.314530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:32.314496    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:32.404336 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:32.404306    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f65f330-0c39-4391-8de1-8d793b792708" path="/var/lib/kubelet/pods/8f65f330-0c39-4391-8de1-8d793b792708/volumes"
Oct 30 06:10:32.554758 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:10:32.554714    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:10:32.937651 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:32.937622    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:33.340087 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:33.340007    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:35.066647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:35.065611    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:36.729050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:36.729013    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:37.442927 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.442897    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:10:37.511423 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.511386    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-certs\") pod \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") "
Oct 30 06:10:37.511630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.511440    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istiod-ca-cert\") pod \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") "
Oct 30 06:10:37.511630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.511480    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-token\" (UniqueName: \"kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-token\") pod \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") "
Oct 30 06:10:37.511630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.511526    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-socket\") pod \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") "
Oct 30 06:10:37.511809 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.511667    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-envoy\") pod \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") "
Oct 30 06:10:37.511809 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.511739    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-certs" pod "aab2742b-6ca0-4d4e-981c-6525b3e7b26d" (UID: "aab2742b-6ca0-4d4e-981c-6525b3e7b26d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:37.511809 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.511748    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-podinfo\") pod \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") "
Oct 30 06:10:37.511971 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.511832    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-data\") pod \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") "
Oct 30 06:10:37.511971 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.511896    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-kube-api-access-rnrxm\" (UniqueName: \"kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-kube-api-access-rnrxm\") pod \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") "
Oct 30 06:10:37.512068 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.511965    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-credential-socket\") pod \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\" (UID: \"aab2742b-6ca0-4d4e-981c-6525b3e7b26d\") "
Oct 30 06:10:37.512068 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.512007    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istiod-ca-cert" pod "aab2742b-6ca0-4d4e-981c-6525b3e7b26d" (UID: "aab2742b-6ca0-4d4e-981c-6525b3e7b26d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:10:37.512196 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.512172    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-socket" pod "aab2742b-6ca0-4d4e-981c-6525b3e7b26d" (UID: "aab2742b-6ca0-4d4e-981c-6525b3e7b26d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:37.512415 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.512383    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-credential-socket" pod "aab2742b-6ca0-4d4e-981c-6525b3e7b26d" (UID: "aab2742b-6ca0-4d4e-981c-6525b3e7b26d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:37.512569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.512557    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-data" pod "aab2742b-6ca0-4d4e-981c-6525b3e7b26d" (UID: "aab2742b-6ca0-4d4e-981c-6525b3e7b26d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:37.512790 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.512775    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:37.512878 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.512797    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:37.512878 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.512809    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:37.512878 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.512821    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:37.512878 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.512838    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:37.515396 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.515348    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-kube-api-access-rnrxm" pod "aab2742b-6ca0-4d4e-981c-6525b3e7b26d" (UID: "aab2742b-6ca0-4d4e-981c-6525b3e7b26d"). InnerVolumeSpecName "kube-api-access-rnrxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:37.515760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.515733    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-token" pod "aab2742b-6ca0-4d4e-981c-6525b3e7b26d" (UID: "aab2742b-6ca0-4d4e-981c-6525b3e7b26d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:37.517453 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.517425    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-podinfo" pod "aab2742b-6ca0-4d4e-981c-6525b3e7b26d" (UID: "aab2742b-6ca0-4d4e-981c-6525b3e7b26d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:10:37.517453 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.517435    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-envoy" pod "aab2742b-6ca0-4d4e-981c-6525b3e7b26d" (UID: "aab2742b-6ca0-4d4e-981c-6525b3e7b26d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:10:37.614017 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.613978    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:37.614017 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.614010    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:37.614017 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.614019    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:37.614265 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.614028    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnrxm\" (UniqueName: \"kubernetes.io/projected/aab2742b-6ca0-4d4e-981c-6525b3e7b26d-kube-api-access-rnrxm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:37.784379 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.784298    5066 generic.go:356] "Generic (PLEG): container finished" podID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" containerID="62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462" exitCode=0
Oct 30 06:10:37.784379 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.784367    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"
Oct 30 06:10:37.784863 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.784384    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" event={"ID":"aab2742b-6ca0-4d4e-981c-6525b3e7b26d","Type":"ContainerDied","Data":"62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462"}
Oct 30 06:10:37.784863 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.784466    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt" event={"ID":"aab2742b-6ca0-4d4e-981c-6525b3e7b26d","Type":"ContainerDied","Data":"4d3ee8c0881d518745d09814571bdabb180142a2e1a9c83463dca4ec31fde244"}
Oct 30 06:10:37.784863 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.784491    5066 scope.go:122] "RemoveContainer" containerID="62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462"
Oct 30 06:10:37.797875 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.797852    5066 scope.go:122] "RemoveContainer" containerID="62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462"
Oct 30 06:10:37.798212 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:10:37.798191    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462\": container with ID starting with 62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462 not found: ID does not exist" containerID="62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462"
Oct 30 06:10:37.798281 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.798226    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462"} err="failed to get container status \"62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462\": rpc error: code = NotFound desc = could not find container \"62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462\": container with ID starting with 62edd84ac9d9da31961ce406d1fa1422d2d4b9cb0e469877e138928399a91462 not found: ID does not exist"
Oct 30 06:10:37.844418 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.844380    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"]
Oct 30 06:10:37.847797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:37.847771    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c1d09f-openshift-default-67b97686fc-mg9rt"]
Oct 30 06:10:38.370551 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:38.369775    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:38.404887 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:38.404850    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" path="/var/lib/kubelet/pods/aab2742b-6ca0-4d4e-981c-6525b3e7b26d/volumes"
Oct 30 06:10:39.862255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:39.862219    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:41.432435 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:41.432397    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:42.228314 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.228280    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"]
Oct 30 06:10:42.229120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229089    5066 cpu_manager.go:414] "containerMap: removing container" podUID="40e57278-1ad4-4d16-b8be-fa1f992467ae" containerName="istio-proxy"
Oct 30 06:10:42.229120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229119    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40e57278-1ad4-4d16-b8be-fa1f992467ae" containerName="istio-proxy"
Oct 30 06:10:42.229320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229137    5066 cpu_manager.go:414] "containerMap: removing container" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy"
Oct 30 06:10:42.229320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229144    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy"
Oct 30 06:10:42.229320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229154    5066 cpu_manager.go:414] "containerMap: removing container" podUID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" containerName="istio-proxy"
Oct 30 06:10:42.229320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229161    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" containerName="istio-proxy"
Oct 30 06:10:42.229320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229204    5066 cpu_manager.go:414] "containerMap: removing container" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy"
Oct 30 06:10:42.229320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229211    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy"
Oct 30 06:10:42.229320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229228    5066 cpu_manager.go:414] "containerMap: removing container" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy"
Oct 30 06:10:42.229320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229234    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy"
Oct 30 06:10:42.229320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229257    5066 cpu_manager.go:414] "containerMap: removing container" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerName="istio-proxy"
Oct 30 06:10:42.229320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229265    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerName="istio-proxy"
Oct 30 06:10:42.229874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229428    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="40e57278-1ad4-4d16-b8be-fa1f992467ae" containerName="istio-proxy"
Oct 30 06:10:42.229874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229447    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="204b23d9-e64b-475c-be79-4a3746371200" containerName="istio-proxy"
Oct 30 06:10:42.229874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229457    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="aab2742b-6ca0-4d4e-981c-6525b3e7b26d" containerName="istio-proxy"
Oct 30 06:10:42.229874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229473    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd59ca3c-8f24-4f58-bd47-01e6eae62873" containerName="istio-proxy"
Oct 30 06:10:42.229874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229483    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f65f330-0c39-4391-8de1-8d793b792708" containerName="istio-proxy"
Oct 30 06:10:42.229874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.229496    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="2736e286-37f0-4c52-9e22-eab3c3b52b5e" containerName="istio-proxy"
Oct 30 06:10:42.235146 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.235121    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.254298 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.254250    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-cfe14f-openshift-default-dockercfg-hbfwm\""
Oct 30 06:10:42.258665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.258632    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"]
Oct 30 06:10:42.378427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.378391    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcr2t\" (UniqueName: \"kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-kube-api-access-wcr2t\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.378678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.378444    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-socket\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.378678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.378469    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-envoy\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.378678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.378619    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-token\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.378678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.378650    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-data\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.378915 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.378777    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/173f259b-bfdc-465f-a4f5-b33e866932e2-istiod-ca-cert\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.378915 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.378870    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-credential-socket\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.379013 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.378999    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-podinfo\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.379050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.379035    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-certs\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.480012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.479924    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcr2t\" (UniqueName: \"kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-kube-api-access-wcr2t\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.480012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.479978    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-socket\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.480012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.480002    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-envoy\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.480823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.480030    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-token\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.480823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.480047    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-data\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.480823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.480105    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/173f259b-bfdc-465f-a4f5-b33e866932e2-istiod-ca-cert\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.480823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.480171    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-credential-socket\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.480823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.480308    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-podinfo\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.480823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.480357    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-certs\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.480823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.480548    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-data\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.481230 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.481003    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/173f259b-bfdc-465f-a4f5-b33e866932e2-istiod-ca-cert\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.481299 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.481227    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-socket\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.481299 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.481273    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-certs\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.481412 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.481344    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-credential-socket\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.482773 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.482749    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-envoy\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.483467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.483447    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-podinfo\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.506559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.506521    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-token\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.506730 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.506632    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcr2t\" (UniqueName: \"kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-kube-api-access-wcr2t\") pod \"e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") " pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.551978 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.551931    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:42.894573 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:10:42.888402    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173f259b_bfdc_465f_a4f5_b33e866932e2.slice/crio-2e55f47e28aba482946e858adf3ab8b83af49c5e96afe3967646dc42fc68eb49 WatchSource:0}: Error finding container 2e55f47e28aba482946e858adf3ab8b83af49c5e96afe3967646dc42fc68eb49: Status 404 returned error can't find the container with id 2e55f47e28aba482946e858adf3ab8b83af49c5e96afe3967646dc42fc68eb49
Oct 30 06:10:42.894573 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.893339    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"]
Oct 30 06:10:42.901792 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.901747    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:10:42.901891 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.901841    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:10:42.901934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.901887    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:10:42.990516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:42.990485    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:43.150900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:43.150822    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:43.497572 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:43.497536    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:43.668781 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:43.668751    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:43.831758 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:43.831662    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr" event={"ID":"173f259b-bfdc-465f-a4f5-b33e866932e2","Type":"ContainerStarted","Data":"83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a"}
Oct 30 06:10:43.831758 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:43.831725    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr" event={"ID":"173f259b-bfdc-465f-a4f5-b33e866932e2","Type":"ContainerStarted","Data":"2e55f47e28aba482946e858adf3ab8b83af49c5e96afe3967646dc42fc68eb49"}
Oct 30 06:10:43.946924 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:43.946842    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr" podStartSLOduration=1.94682201 podStartE2EDuration="1.94682201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:10:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:10:43.94239661 +0000 UTC m=+2347.687681941" watchObservedRunningTime="2026-10-30 06:10:43.94682201 +0000 UTC m=+2347.692107345"
Oct 30 06:10:44.552895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:44.552866    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:44.554813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:44.554786    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Oct 30 06:10:44.554942 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:44.554851    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr" podUID="173f259b-bfdc-465f-a4f5-b33e866932e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Oct 30 06:10:44.820824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:44.820732    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:45.553056 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:45.553025    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Oct 30 06:10:45.553476 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:45.553078    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr" podUID="173f259b-bfdc-465f-a4f5-b33e866932e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Oct 30 06:10:46.215392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:46.215344    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"]
Oct 30 06:10:46.215668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:46.215641    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6" podUID="3df0470a-ec9f-48fe-85bf-826b4ac4f6b8" containerName="maas-api" containerID="cri-o://001c087aeb50830b1a330f31bd7e2d0dc6a22c830455384d410876be1f1ea016" gracePeriod=30
Oct 30 06:10:46.360561 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:46.360523    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:46.563682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:46.553136    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Oct 30 06:10:46.708520 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:46.708482    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr" podUID="173f259b-bfdc-465f-a4f5-b33e866932e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Oct 30 06:10:46.977680 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:46.976922    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:46.981502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:46.981340    5066 generic.go:356] "Generic (PLEG): container finished" podID="3df0470a-ec9f-48fe-85bf-826b4ac4f6b8" containerID="001c087aeb50830b1a330f31bd7e2d0dc6a22c830455384d410876be1f1ea016" exitCode=0
Oct 30 06:10:46.981502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:46.981411    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6" event={"ID":"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8","Type":"ContainerDied","Data":"001c087aeb50830b1a330f31bd7e2d0dc6a22c830455384d410876be1f1ea016"}
Oct 30 06:10:46.981502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:46.981460    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6" event={"ID":"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8","Type":"ContainerDied","Data":"19a3098f3feea7d3329908c58938512c85afbe1998bc37fe1edb1f212271c1ca"}
Oct 30 06:10:46.981502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:46.981483    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19a3098f3feea7d3329908c58938512c85afbe1998bc37fe1edb1f212271c1ca"
Oct 30 06:10:46.986382 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:46.986358    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:10:47.083015 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.082924    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-openshift-service-ca\") pod \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") "
Oct 30 06:10:47.083015 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.082973    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-metrics-certs\") pod \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") "
Oct 30 06:10:47.083220 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.083139    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-maas-api-tls\") pod \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") "
Oct 30 06:10:47.083220 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.083211    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-kube-api-access-4lcz6\" (UniqueName: \"kubernetes.io/projected/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-kube-api-access-4lcz6\") pod \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\" (UID: \"3df0470a-ec9f-48fe-85bf-826b4ac4f6b8\") "
Oct 30 06:10:47.084376 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.084182    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-openshift-service-ca" pod "3df0470a-ec9f-48fe-85bf-826b4ac4f6b8" (UID: "3df0470a-ec9f-48fe-85bf-826b4ac4f6b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:10:47.096286 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.096237    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-metrics-certs" pod "3df0470a-ec9f-48fe-85bf-826b4ac4f6b8" (UID: "3df0470a-ec9f-48fe-85bf-826b4ac4f6b8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:10:47.098369 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.098335    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-maas-api-tls" pod "3df0470a-ec9f-48fe-85bf-826b4ac4f6b8" (UID: "3df0470a-ec9f-48fe-85bf-826b4ac4f6b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:10:47.100635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.099451    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-kube-api-access-4lcz6" pod "3df0470a-ec9f-48fe-85bf-826b4ac4f6b8" (UID: "3df0470a-ec9f-48fe-85bf-826b4ac4f6b8"). InnerVolumeSpecName "kube-api-access-4lcz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:10:47.152617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.148055    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:47.185232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.185194    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:47.185232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.185225    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lcz6\" (UniqueName: \"kubernetes.io/projected/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-kube-api-access-4lcz6\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:47.185232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.185235    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:47.185477 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.185246    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:10:47.321618 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.315143    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:47.553613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.552985    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Oct 30 06:10:47.553613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.553046    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr" podUID="173f259b-bfdc-465f-a4f5-b33e866932e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Oct 30 06:10:47.618583 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.618544    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:47.802502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.802471    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:47.988207 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:47.988172    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"
Oct 30 06:10:48.067709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:48.067675    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"]
Oct 30 06:10:48.071464 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:48.071438    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7df9fa-6b874887b6-dp6w6"]
Oct 30 06:10:48.404869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:48.404840    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3df0470a-ec9f-48fe-85bf-826b4ac4f6b8" path="/var/lib/kubelet/pods/3df0470a-ec9f-48fe-85bf-826b4ac4f6b8/volumes"
Oct 30 06:10:48.554961 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:48.554922    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Oct 30 06:10:48.555142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:48.554991    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr" podUID="173f259b-bfdc-465f-a4f5-b33e866932e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Oct 30 06:10:49.557618 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:49.556978    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Oct 30 06:10:49.557618 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:49.557039    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr" podUID="173f259b-bfdc-465f-a4f5-b33e866932e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Oct 30 06:10:49.663622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:49.663079    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:50.663763 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:50.663582    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:50.664274 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:50.664257    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:50.665120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:50.665089    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:10:51.278048 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:51.278011    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:52.831550 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:52.831515    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:52.973195 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:52.973157    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"]
Oct 30 06:10:52.973900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:52.973878    5066 cpu_manager.go:414] "containerMap: removing container" podUID="3df0470a-ec9f-48fe-85bf-826b4ac4f6b8" containerName="maas-api"
Oct 30 06:10:52.973900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:52.973899    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3df0470a-ec9f-48fe-85bf-826b4ac4f6b8" containerName="maas-api"
Oct 30 06:10:52.974114 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:52.974100    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="3df0470a-ec9f-48fe-85bf-826b4ac4f6b8" containerName="maas-api"
Oct 30 06:10:52.978853 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:52.978833    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.000112 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.000078    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-d3e7e6-openshift-default-dockercfg-vchrg\""
Oct 30 06:10:53.027712 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.027676    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"]
Oct 30 06:10:53.031579 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.031553    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"]
Oct 30 06:10:53.036415 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.036389    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.061151 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.061118    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"]
Oct 30 06:10:53.090556 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.090470    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b307c327-0d78-4749-99e4-30471e9410c0-istiod-ca-cert\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.090556 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.090510    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-certs\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.090824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.090575    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-envoy\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.090824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.090618    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcpjx\" (UniqueName: \"kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-kube-api-access-qcpjx\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.090824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.090696    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-envoy\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.090824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.090761    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-podinfo\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.090824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.090785    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-credential-socket\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.091086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.090865    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e84b77a-82d5-4863-82c8-dee9669b54b2-istiod-ca-cert\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.091086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.090903    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-credential-socket\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.091086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.090936    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-data\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.091086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.090986    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-data\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.091086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.091029    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-token\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.091086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.091070    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b307c327-0d78-4749-99e4-30471e9410c0-istio-podinfo\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.091397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.091093    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-socket\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.091397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.091181    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tl2w\" (UniqueName: \"kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-kube-api-access-7tl2w\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.091397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.091215    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-certs\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.091397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.091308    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-socket\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.091397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.091368    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-istio-token\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.111840 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.111813    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-0d64c9-openshift-default-dockercfg-zxnwz\""
Oct 30 06:10:53.193010 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.192973    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-data\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.193227 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193034    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-data\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.193227 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193082    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-token\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.193227 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193134    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b307c327-0d78-4749-99e4-30471e9410c0-istio-podinfo\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.193227 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193158    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-socket\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.193452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193234    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tl2w\" (UniqueName: \"kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-kube-api-access-7tl2w\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.193452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193266    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-certs\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.193452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193344    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-socket\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.193452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193401    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-istio-token\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.193452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193410    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-data\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.193452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193430    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b307c327-0d78-4749-99e4-30471e9410c0-istiod-ca-cert\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.193837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193460    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-data\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.193837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193483    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-certs\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.193837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193512    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-envoy\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.193837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193536    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-socket\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.193837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193552    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcpjx\" (UniqueName: \"kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-kube-api-access-qcpjx\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.193837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193627    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-envoy\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.193837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193635    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-certs\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.193837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193703    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-socket\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.193837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193740    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-podinfo\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.193837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193778    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-credential-socket\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.194345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193856    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e84b77a-82d5-4863-82c8-dee9669b54b2-istiod-ca-cert\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.194345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193871    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-certs\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.194345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.193897    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-credential-socket\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.194345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.194131    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-credential-socket\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.194345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.194272    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-credential-socket\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.194565 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.194517    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e84b77a-82d5-4863-82c8-dee9669b54b2-istiod-ca-cert\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.194690 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.194669    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b307c327-0d78-4749-99e4-30471e9410c0-istiod-ca-cert\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.195989 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.195969    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-envoy\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.196476 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.196458    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b307c327-0d78-4749-99e4-30471e9410c0-istio-podinfo\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.196545 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.196504    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-envoy\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.196734 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.196715    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-podinfo\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.219402 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.219372    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-istio-token\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.219620 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.219573    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tl2w\" (UniqueName: \"kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-kube-api-access-7tl2w\") pod \"e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") " pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.220941 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.220879    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-token\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.221465 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.221444    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcpjx\" (UniqueName: \"kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-kube-api-access-qcpjx\") pod \"e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") " pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.293574 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.293542    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:53.357277 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.357250    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:53.619665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.619638    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"]
Oct 30 06:10:53.619803 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:10:53.619781    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb307c327_0d78_4749_99e4_30471e9410c0.slice/crio-8615c7ae68b6e152aa4522d7be5ded02e8580127ea25f09da144665e527332ac WatchSource:0}: Error finding container 8615c7ae68b6e152aa4522d7be5ded02e8580127ea25f09da144665e527332ac: Status 404 returned error can't find the container with id 8615c7ae68b6e152aa4522d7be5ded02e8580127ea25f09da144665e527332ac
Oct 30 06:10:53.622730 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.622697    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:10:53.622841 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.622764    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:10:53.622841 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.622802    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:10:53.682785 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.682748    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"]
Oct 30 06:10:53.682902 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:10:53.682823    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e84b77a_82d5_4863_82c8_dee9669b54b2.slice/crio-02efca8edb4b91b9ce5a462c5124d1baf3b4733d64d80e08653da59e4eb98f33 WatchSource:0}: Error finding container 02efca8edb4b91b9ce5a462c5124d1baf3b4733d64d80e08653da59e4eb98f33: Status 404 returned error can't find the container with id 02efca8edb4b91b9ce5a462c5124d1baf3b4733d64d80e08653da59e4eb98f33
Oct 30 06:10:53.686185 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.686147    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:10:53.686284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.686214    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:10:53.686284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:53.686244    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:10:54.070669 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.066665    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" event={"ID":"9e84b77a-82d5-4863-82c8-dee9669b54b2","Type":"ContainerStarted","Data":"a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32"}
Oct 30 06:10:54.070669 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.067004    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" event={"ID":"9e84b77a-82d5-4863-82c8-dee9669b54b2","Type":"ContainerStarted","Data":"02efca8edb4b91b9ce5a462c5124d1baf3b4733d64d80e08653da59e4eb98f33"}
Oct 30 06:10:54.073987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.073955    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" event={"ID":"b307c327-0d78-4749-99e4-30471e9410c0","Type":"ContainerStarted","Data":"75a17c44e1eaddfd91d1f0464e91ac21892e0d01699c02016adef8a28d09072f"}
Oct 30 06:10:54.074094 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.074020    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" event={"ID":"b307c327-0d78-4749-99e4-30471e9410c0","Type":"ContainerStarted","Data":"8615c7ae68b6e152aa4522d7be5ded02e8580127ea25f09da144665e527332ac"}
Oct 30 06:10:54.120973 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.120925    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" podStartSLOduration=1.120911015 podStartE2EDuration="1.120911015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:10:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:10:54.116779135 +0000 UTC m=+2357.862064506" watchObservedRunningTime="2026-10-30 06:10:54.120911015 +0000 UTC m=+2357.866196346"
Oct 30 06:10:54.166964 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.166912    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" podStartSLOduration=2.166893 podStartE2EDuration="2.166893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:10:54.165359836 +0000 UTC m=+2357.910645168" watchObservedRunningTime="2026-10-30 06:10:54.166893 +0000 UTC m=+2357.912178333"
Oct 30 06:10:54.294245 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.294209    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:10:54.295833 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.295805    5066 patch_prober.go:29] interesting pod/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 30 06:10:54.295956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.295861    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" podUID="b307c327-0d78-4749-99e4-30471e9410c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 30 06:10:54.358399 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.358295    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:10:54.361753 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.361717    5066 patch_prober.go:29] interesting pod/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Oct 30 06:10:54.361947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.361776    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" podUID="9e84b77a-82d5-4863-82c8-dee9669b54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Oct 30 06:10:54.745400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:54.745366    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:55.294324 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:55.294289    5066 patch_prober.go:29] interesting pod/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 30 06:10:55.294855 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:55.294337    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" podUID="b307c327-0d78-4749-99e4-30471e9410c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 30 06:10:55.354636 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:55.354577    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:55.358626 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:55.358572    5066 patch_prober.go:29] interesting pod/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Oct 30 06:10:55.358840 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:55.358816    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" podUID="9e84b77a-82d5-4863-82c8-dee9669b54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Oct 30 06:10:55.521090 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:55.521056    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:55.777623 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:55.768264    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"]
Oct 30 06:10:55.778136 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:55.778100    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerName="istio-proxy" containerID="cri-o://a179d98aa61e46cf9229051e14fffce4992da27e0c70971bd94aa61a8ae9214c" gracePeriod=30
Oct 30 06:10:56.129334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:56.129248    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:56.307854 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:56.307803    5066 patch_prober.go:29] interesting pod/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 30 06:10:56.308511 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:56.308482    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" podUID="b307c327-0d78-4749-99e4-30471e9410c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 30 06:10:56.432673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:56.394882    5066 patch_prober.go:29] interesting pod/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Oct 30 06:10:56.432960 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:56.432928    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" podUID="9e84b77a-82d5-4863-82c8-dee9669b54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Oct 30 06:10:56.504612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:56.504166    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 06:10:56.950614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:56.946280    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:57.106777 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:57.106744    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:57.294102 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:57.294022    5066 patch_prober.go:29] interesting pod/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 30 06:10:57.294102 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:57.294082    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" podUID="b307c327-0d78-4749-99e4-30471e9410c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 30 06:10:57.358948 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:57.358916    5066 patch_prober.go:29] interesting pod/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Oct 30 06:10:57.359537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:57.359506    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" podUID="9e84b77a-82d5-4863-82c8-dee9669b54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Oct 30 06:10:57.395398 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:57.395368    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:58.017615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:58.014522    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:58.142107 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:58.142074    5066 patch_prober.go:29] interesting pod/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:10:58.142313 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:58.142293    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:10:58.306692 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:58.300911    5066 patch_prober.go:29] interesting pod/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 30 06:10:58.306692 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:58.300966    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" podUID="b307c327-0d78-4749-99e4-30471e9410c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 30 06:10:58.363864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:58.363813    5066 patch_prober.go:29] interesting pod/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Oct 30 06:10:58.365421 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:58.365388    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" podUID="9e84b77a-82d5-4863-82c8-dee9669b54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Oct 30 06:10:58.655677 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:58.653298    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:10:59.305617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:59.304876    5066 patch_prober.go:29] interesting pod/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 30 06:10:59.305617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:59.304932    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" podUID="b307c327-0d78-4749-99e4-30471e9410c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 30 06:10:59.370616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:59.364029    5066 patch_prober.go:29] interesting pod/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Oct 30 06:10:59.370616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:59.364090    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" podUID="9e84b77a-82d5-4863-82c8-dee9669b54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Oct 30 06:10:59.639154 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:10:59.639068    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:00.294163 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:00.294121    5066 patch_prober.go:29] interesting pod/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 30 06:11:00.294384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:00.294186    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" podUID="b307c327-0d78-4749-99e4-30471e9410c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 30 06:11:00.360058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:00.360023    5066 patch_prober.go:29] interesting pod/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Oct 30 06:11:00.360317 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:00.360292    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" podUID="9e84b77a-82d5-4863-82c8-dee9669b54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Oct 30 06:11:01.316605 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.314905    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:11:01.316605 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.316408    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:11:01.322944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.322854    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:11:01.385702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.385678    5066 generic.go:356] "Generic (PLEG): container finished" podID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerID="a179d98aa61e46cf9229051e14fffce4992da27e0c70971bd94aa61a8ae9214c" exitCode=0
Oct 30 06:11:01.386875 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.386853    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" event={"ID":"09e2d559-d3fe-4609-ac77-04723d70ab06","Type":"ContainerDied","Data":"a179d98aa61e46cf9229051e14fffce4992da27e0c70971bd94aa61a8ae9214c"}
Oct 30 06:11:01.550768 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.550734    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:01.572166 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.572133    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:11:01.572579 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.572555    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:11:01.573530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.573512    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:11:01.590175 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.590150    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:11:01.750618 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.750111    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-data\") pod \"09e2d559-d3fe-4609-ac77-04723d70ab06\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") "
Oct 30 06:11:01.750618 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.750185    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-token\" (UniqueName: \"kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-token\") pod \"09e2d559-d3fe-4609-ac77-04723d70ab06\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") "
Oct 30 06:11:01.773613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.766036    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-data" pod "09e2d559-d3fe-4609-ac77-04723d70ab06" (UID: "09e2d559-d3fe-4609-ac77-04723d70ab06"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:01.773810 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.750234    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-socket\") pod \"09e2d559-d3fe-4609-ac77-04723d70ab06\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") "
Oct 30 06:11:01.773810 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.773724    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-podinfo\") pod \"09e2d559-d3fe-4609-ac77-04723d70ab06\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") "
Oct 30 06:11:01.773948 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.773821    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-envoy\") pod \"09e2d559-d3fe-4609-ac77-04723d70ab06\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") "
Oct 30 06:11:01.773948 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.773933    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-certs\") pod \"09e2d559-d3fe-4609-ac77-04723d70ab06\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") "
Oct 30 06:11:01.774063 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.773994    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-kube-api-access-jwvbt\" (UniqueName: \"kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-kube-api-access-jwvbt\") pod \"09e2d559-d3fe-4609-ac77-04723d70ab06\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") "
Oct 30 06:11:01.774124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.774087    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-credential-socket\") pod \"09e2d559-d3fe-4609-ac77-04723d70ab06\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") "
Oct 30 06:11:01.774192 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.774157    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/09e2d559-d3fe-4609-ac77-04723d70ab06-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09e2d559-d3fe-4609-ac77-04723d70ab06-istiod-ca-cert\") pod \"09e2d559-d3fe-4609-ac77-04723d70ab06\" (UID: \"09e2d559-d3fe-4609-ac77-04723d70ab06\") "
Oct 30 06:11:01.782612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.774951    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:01.782612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.781709    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-token" pod "09e2d559-d3fe-4609-ac77-04723d70ab06" (UID: "09e2d559-d3fe-4609-ac77-04723d70ab06"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:11:01.782612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.781975    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-certs" pod "09e2d559-d3fe-4609-ac77-04723d70ab06" (UID: "09e2d559-d3fe-4609-ac77-04723d70ab06"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:01.785555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.785527    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-socket" pod "09e2d559-d3fe-4609-ac77-04723d70ab06" (UID: "09e2d559-d3fe-4609-ac77-04723d70ab06"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:01.791116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.790035    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09e2d559-d3fe-4609-ac77-04723d70ab06-istiod-ca-cert" pod "09e2d559-d3fe-4609-ac77-04723d70ab06" (UID: "09e2d559-d3fe-4609-ac77-04723d70ab06"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:11:01.791116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.790107    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-credential-socket" pod "09e2d559-d3fe-4609-ac77-04723d70ab06" (UID: "09e2d559-d3fe-4609-ac77-04723d70ab06"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:01.791116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.791015    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:01.828888 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.828851    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-kube-api-access-jwvbt" pod "09e2d559-d3fe-4609-ac77-04723d70ab06" (UID: "09e2d559-d3fe-4609-ac77-04723d70ab06"). InnerVolumeSpecName "kube-api-access-jwvbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:11:01.837310 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.836852    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-envoy" pod "09e2d559-d3fe-4609-ac77-04723d70ab06" (UID: "09e2d559-d3fe-4609-ac77-04723d70ab06"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:01.837310 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.837031    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-podinfo" pod "09e2d559-d3fe-4609-ac77-04723d70ab06" (UID: "09e2d559-d3fe-4609-ac77-04723d70ab06"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:11:01.889613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.885045    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:01.889613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.885079    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:01.889613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.885093    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:01.889613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.885118    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:01.889613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.885136    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:01.889613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.885151    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwvbt\" (UniqueName: \"kubernetes.io/projected/09e2d559-d3fe-4609-ac77-04723d70ab06-kube-api-access-jwvbt\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:01.889613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.885167    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09e2d559-d3fe-4609-ac77-04723d70ab06-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:01.889613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:01.885183    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09e2d559-d3fe-4609-ac77-04723d70ab06-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:02.394652 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:02.394624    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"
Oct 30 06:11:02.403885 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:02.403859    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8" event={"ID":"09e2d559-d3fe-4609-ac77-04723d70ab06","Type":"ContainerDied","Data":"468aa9e4255cd34cd4ba893b52de25827a0e0f79fae2849e18b34e67ec341d7a"}
Oct 30 06:11:02.403977 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:02.403895    5066 scope.go:122] "RemoveContainer" containerID="a179d98aa61e46cf9229051e14fffce4992da27e0c70971bd94aa61a8ae9214c"
Oct 30 06:11:02.455688 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:02.455616    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"]
Oct 30 06:11:02.469422 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:02.469391    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-7df9fa-openshift-default-b67f7d48f-t7hr8"]
Oct 30 06:11:02.553664 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:11:02.553629    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:11:03.114350 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:03.114319    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:03.423864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:03.423828    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:03.768060 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:03.767953    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:04.403944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:04.403913    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" path="/var/lib/kubelet/pods/09e2d559-d3fe-4609-ac77-04723d70ab06/volumes"
Oct 30 06:11:05.166734 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:05.166681    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:05.590890 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:05.590764    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:06.718337 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:06.718280    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:06.883399 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:06.883355    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:07.236635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:07.236392    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:08.910949 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:08.910898    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:09.419541 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:09.419503    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:10.189072 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:10.189039    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:10.715353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:10.715317    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:10.881396 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:10.881346    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:11.347995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:11.347964    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:12.543084 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:12.543049    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:14.016047 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:14.016010    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:14.728619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:14.728569    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:15.051448 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:15.051346    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:15.397718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:15.397644    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:16.799173 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:16.799089    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:18.553269 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:18.553240    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:18.724945 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:18.724912    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:18.973475 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:18.973440    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"]
Oct 30 06:11:18.973729 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:18.973703    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv" podUID="1a325b74-cd9e-4678-a445-56cb549a12fb" containerName="istio-proxy" containerID="cri-o://b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe" gracePeriod=30
Oct 30 06:11:19.162862 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:19.162828    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:20.030091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.030059    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"]
Oct 30 06:11:20.031036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.031017    5066 cpu_manager.go:414] "containerMap: removing container" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerName="istio-proxy"
Oct 30 06:11:20.031036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.031036    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerName="istio-proxy"
Oct 30 06:11:20.031196 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.031185    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="09e2d559-d3fe-4609-ac77-04723d70ab06" containerName="istio-proxy"
Oct 30 06:11:20.035800 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.035778    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.056751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.056721    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"]
Oct 30 06:11:20.065550 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.065519    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-03d349-openshift-default-dockercfg-2zmxp\""
Oct 30 06:11:20.078354 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.078316    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ff0902-2fa3-4267-b865-3b301828a993-istiod-ca-cert\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.078563 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.078381    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-certs\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.078563 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.078492    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-data\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.078766 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.078624    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-envoy\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.078766 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.078663    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-credential-socket\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.078882 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.078838    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ff0902-2fa3-4267-b865-3b301828a993-istio-podinfo\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.078944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.078897    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-istio-token\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.078944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.078924    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wm2w\" (UniqueName: \"kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-kube-api-access-4wm2w\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.079056 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.078975    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-socket\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.180147 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.180094    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-istio-token\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.180147 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.180152    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wm2w\" (UniqueName: \"kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-kube-api-access-4wm2w\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.180400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.180207    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-socket\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.180400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.180296    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ff0902-2fa3-4267-b865-3b301828a993-istiod-ca-cert\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.180400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.180336    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-certs\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.180555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.180449    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-data\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.180628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.180567    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-envoy\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.180702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.180630    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-credential-socket\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.180702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.180691    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ff0902-2fa3-4267-b865-3b301828a993-istio-podinfo\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.180808 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.180780    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-socket\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.180868 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.180827    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-certs\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.180930 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.180889    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-data\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.181141 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.181108    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-credential-socket\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.181299 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.181243    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ff0902-2fa3-4267-b865-3b301828a993-istiod-ca-cert\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.183537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.183513    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-envoy\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.187289 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.187259    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ff0902-2fa3-4267-b865-3b301828a993-istio-podinfo\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.207371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.207336    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wm2w\" (UniqueName: \"kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-kube-api-access-4wm2w\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.207823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.207800    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-istio-token\") pod \"e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") " pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.248671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.248610    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:20.350742 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.350655    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:20.687009 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:11:20.686970    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ff0902_2fa3_4267_b865_3b301828a993.slice/crio-7f3d8fbd6da92c878e7ba980a27b5574b8119ee5d10071b003bda9fcb6e48b44 WatchSource:0}: Error finding container 7f3d8fbd6da92c878e7ba980a27b5574b8119ee5d10071b003bda9fcb6e48b44: Status 404 returned error can't find the container with id 7f3d8fbd6da92c878e7ba980a27b5574b8119ee5d10071b003bda9fcb6e48b44
Oct 30 06:11:20.687202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.687177    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"]
Oct 30 06:11:20.689721 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.689687    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:11:20.689826 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.689764    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:11:20.689899 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.689829    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:11:20.760605 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:20.760563    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:21.650615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:21.649868    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw" event={"ID":"d7ff0902-2fa3-4267-b865-3b301828a993","Type":"ContainerStarted","Data":"30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3"}
Oct 30 06:11:21.650615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:21.649940    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw" event={"ID":"d7ff0902-2fa3-4267-b865-3b301828a993","Type":"ContainerStarted","Data":"7f3d8fbd6da92c878e7ba980a27b5574b8119ee5d10071b003bda9fcb6e48b44"}
Oct 30 06:11:21.735619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:21.735417    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw" podStartSLOduration=2.7353948900000002 podStartE2EDuration="2.73539489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:11:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:11:21.715088711 +0000 UTC m=+2385.460374043" watchObservedRunningTime="2026-10-30 06:11:21.73539489 +0000 UTC m=+2385.480680224"
Oct 30 06:11:22.071614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:22.069492    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:22.351674 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:22.351217    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:22.352818 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:22.352788    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused" start-of-body=
Oct 30 06:11:22.352956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:22.352851    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw" podUID="d7ff0902-2fa3-4267-b865-3b301828a993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused"
Oct 30 06:11:22.647167 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:22.647075    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:22.849649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:22.847958    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:22.969441 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:22.969404    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"]
Oct 30 06:11:22.970283 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:22.969683    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" podUID="9e84b77a-82d5-4863-82c8-dee9669b54b2" containerName="istio-proxy" containerID="cri-o://a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32" gracePeriod=30
Oct 30 06:11:23.049614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:23.049288    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:23.351128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:23.351038    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused" start-of-body=
Oct 30 06:11:23.351128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:23.351112    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw" podUID="d7ff0902-2fa3-4267-b865-3b301828a993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused"
Oct 30 06:11:24.145082 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.144995    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:24.359698 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.359578    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused" start-of-body=
Oct 30 06:11:24.359846 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.359752    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw" podUID="d7ff0902-2fa3-4267-b865-3b301828a993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused"
Oct 30 06:11:24.396371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.396338    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:11:24.564428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.564393    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-podinfo\") pod \"1a325b74-cd9e-4678-a445-56cb549a12fb\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") "
Oct 30 06:11:24.564660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.564475    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a325b74-cd9e-4678-a445-56cb549a12fb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a325b74-cd9e-4678-a445-56cb549a12fb-istiod-ca-cert\") pod \"1a325b74-cd9e-4678-a445-56cb549a12fb\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") "
Oct 30 06:11:24.564660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.564538    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-data\") pod \"1a325b74-cd9e-4678-a445-56cb549a12fb\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") "
Oct 30 06:11:24.564660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.564653    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-token\" (UniqueName: \"kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-token\") pod \"1a325b74-cd9e-4678-a445-56cb549a12fb\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") "
Oct 30 06:11:24.564865 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.564675    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-certs\") pod \"1a325b74-cd9e-4678-a445-56cb549a12fb\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") "
Oct 30 06:11:24.564865 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.564734    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-kube-api-access-z5rcb\" (UniqueName: \"kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-kube-api-access-z5rcb\") pod \"1a325b74-cd9e-4678-a445-56cb549a12fb\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") "
Oct 30 06:11:24.564950 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.564861    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-data" pod "1a325b74-cd9e-4678-a445-56cb549a12fb" (UID: "1a325b74-cd9e-4678-a445-56cb549a12fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:24.564950 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.564914    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-envoy\") pod \"1a325b74-cd9e-4678-a445-56cb549a12fb\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") "
Oct 30 06:11:24.565035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.564969    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-credential-socket\") pod \"1a325b74-cd9e-4678-a445-56cb549a12fb\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") "
Oct 30 06:11:24.565035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.564972    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a325b74-cd9e-4678-a445-56cb549a12fb-istiod-ca-cert" pod "1a325b74-cd9e-4678-a445-56cb549a12fb" (UID: "1a325b74-cd9e-4678-a445-56cb549a12fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:11:24.565035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.564989    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-certs" pod "1a325b74-cd9e-4678-a445-56cb549a12fb" (UID: "1a325b74-cd9e-4678-a445-56cb549a12fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:24.565035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.565014    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-socket\") pod \"1a325b74-cd9e-4678-a445-56cb549a12fb\" (UID: \"1a325b74-cd9e-4678-a445-56cb549a12fb\") "
Oct 30 06:11:24.565287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.565244    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-credential-socket" pod "1a325b74-cd9e-4678-a445-56cb549a12fb" (UID: "1a325b74-cd9e-4678-a445-56cb549a12fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:24.565345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.565307    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-socket" pod "1a325b74-cd9e-4678-a445-56cb549a12fb" (UID: "1a325b74-cd9e-4678-a445-56cb549a12fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:24.566168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.566037    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:24.566168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.566064    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:24.566168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.566081    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a325b74-cd9e-4678-a445-56cb549a12fb-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:24.566168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.566095    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:24.566168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.566110    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:24.567319 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.567295    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-envoy" pod "1a325b74-cd9e-4678-a445-56cb549a12fb" (UID: "1a325b74-cd9e-4678-a445-56cb549a12fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:24.567416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.567301    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-kube-api-access-z5rcb" pod "1a325b74-cd9e-4678-a445-56cb549a12fb" (UID: "1a325b74-cd9e-4678-a445-56cb549a12fb"). InnerVolumeSpecName "kube-api-access-z5rcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:11:24.567604 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.567570    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-podinfo" pod "1a325b74-cd9e-4678-a445-56cb549a12fb" (UID: "1a325b74-cd9e-4678-a445-56cb549a12fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:11:24.568002 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.567982    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-token" pod "1a325b74-cd9e-4678-a445-56cb549a12fb" (UID: "1a325b74-cd9e-4678-a445-56cb549a12fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:11:24.667980 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.667941    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:24.667980 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.667974    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:24.667980 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.667985    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:24.668230 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.667995    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5rcb\" (UniqueName: \"kubernetes.io/projected/1a325b74-cd9e-4678-a445-56cb549a12fb-kube-api-access-z5rcb\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:24.697440 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.697403    5066 generic.go:356] "Generic (PLEG): container finished" podID="1a325b74-cd9e-4678-a445-56cb549a12fb" containerID="b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe" exitCode=0
Oct 30 06:11:24.697629 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.697476    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"
Oct 30 06:11:24.697629 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.697486    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv" event={"ID":"1a325b74-cd9e-4678-a445-56cb549a12fb","Type":"ContainerDied","Data":"b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe"}
Oct 30 06:11:24.697629 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.697552    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv" event={"ID":"1a325b74-cd9e-4678-a445-56cb549a12fb","Type":"ContainerDied","Data":"622320e37dc02aaf9c71945b8146e6fc76a05c406a7b65f2d7363145683dacca"}
Oct 30 06:11:24.697629 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.697574    5066 scope.go:122] "RemoveContainer" containerID="b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe"
Oct 30 06:11:24.709253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.709231    5066 scope.go:122] "RemoveContainer" containerID="b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe"
Oct 30 06:11:24.709694 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:11:24.709669    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe\": container with ID starting with b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe not found: ID does not exist" containerID="b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe"
Oct 30 06:11:24.709795 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.709704    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe"} err="failed to get container status \"b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe\": rpc error: code = NotFound desc = could not find container \"b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe\": container with ID starting with b4a8918e9501790b14d2ab8812354c6eb1d542f21627f6d8ae95e680f2b43ffe not found: ID does not exist"
Oct 30 06:11:24.768981 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.768947    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"]
Oct 30 06:11:24.772466 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:24.772446    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-62316c-openshift-default-96996f9c9-wntgv"]
Oct 30 06:11:25.351552 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:25.351520    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused" start-of-body=
Oct 30 06:11:25.352064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:25.351625    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw" podUID="d7ff0902-2fa3-4267-b865-3b301828a993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused"
Oct 30 06:11:26.022116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:26.022081    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:26.351113 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:26.351028    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused" start-of-body=
Oct 30 06:11:26.351276 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:26.351106    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw" podUID="d7ff0902-2fa3-4267-b865-3b301828a993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused"
Oct 30 06:11:26.403365 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:26.403331    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a325b74-cd9e-4678-a445-56cb549a12fb" path="/var/lib/kubelet/pods/1a325b74-cd9e-4678-a445-56cb549a12fb/volumes"
Oct 30 06:11:26.782459 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:26.782416    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:27.242145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:27.242109    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:27.429484 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:27.429430    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:27.430014 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:27.429918    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:27.430894 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:27.430865    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:11:27.773672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:27.773636    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:27.988613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:27.966187    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:28.142275 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.142234    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:28.290205 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.290131    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:11:28.424248 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.424212    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-data\") pod \"9e84b77a-82d5-4863-82c8-dee9669b54b2\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") "
Oct 30 06:11:28.424440 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.424276    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-kube-api-access-qcpjx\" (UniqueName: \"kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-kube-api-access-qcpjx\") pod \"9e84b77a-82d5-4863-82c8-dee9669b54b2\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") "
Oct 30 06:11:28.424440 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.424327    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-token\" (UniqueName: \"kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-token\") pod \"9e84b77a-82d5-4863-82c8-dee9669b54b2\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") "
Oct 30 06:11:28.424440 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.424421    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-credential-socket\") pod \"9e84b77a-82d5-4863-82c8-dee9669b54b2\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") "
Oct 30 06:11:28.424679 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.424451    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-socket\") pod \"9e84b77a-82d5-4863-82c8-dee9669b54b2\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") "
Oct 30 06:11:28.424679 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.424531    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-certs\") pod \"9e84b77a-82d5-4863-82c8-dee9669b54b2\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") "
Oct 30 06:11:28.424679 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.424642    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e84b77a-82d5-4863-82c8-dee9669b54b2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e84b77a-82d5-4863-82c8-dee9669b54b2-istiod-ca-cert\") pod \"9e84b77a-82d5-4863-82c8-dee9669b54b2\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") "
Oct 30 06:11:28.424897 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.424725    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-podinfo\") pod \"9e84b77a-82d5-4863-82c8-dee9669b54b2\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") "
Oct 30 06:11:28.424897 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.424725    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-credential-socket" pod "9e84b77a-82d5-4863-82c8-dee9669b54b2" (UID: "9e84b77a-82d5-4863-82c8-dee9669b54b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:28.425016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.424984    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-certs" pod "9e84b77a-82d5-4863-82c8-dee9669b54b2" (UID: "9e84b77a-82d5-4863-82c8-dee9669b54b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:28.425081 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.425040    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-envoy\") pod \"9e84b77a-82d5-4863-82c8-dee9669b54b2\" (UID: \"9e84b77a-82d5-4863-82c8-dee9669b54b2\") "
Oct 30 06:11:28.425081 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.425054    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e84b77a-82d5-4863-82c8-dee9669b54b2-istiod-ca-cert" pod "9e84b77a-82d5-4863-82c8-dee9669b54b2" (UID: "9e84b77a-82d5-4863-82c8-dee9669b54b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:11:28.425242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.424544    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-data" pod "9e84b77a-82d5-4863-82c8-dee9669b54b2" (UID: "9e84b77a-82d5-4863-82c8-dee9669b54b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:28.426064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.425352    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-socket" pod "9e84b77a-82d5-4863-82c8-dee9669b54b2" (UID: "9e84b77a-82d5-4863-82c8-dee9669b54b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:28.426064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.425916    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:28.426064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.425998    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:28.426064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.426013    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:28.426064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.426027    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:28.426064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.426040    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e84b77a-82d5-4863-82c8-dee9669b54b2-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:28.427486 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.427358    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-kube-api-access-qcpjx" pod "9e84b77a-82d5-4863-82c8-dee9669b54b2" (UID: "9e84b77a-82d5-4863-82c8-dee9669b54b2"). InnerVolumeSpecName "kube-api-access-qcpjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:11:28.427486 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.427397    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-token" pod "9e84b77a-82d5-4863-82c8-dee9669b54b2" (UID: "9e84b77a-82d5-4863-82c8-dee9669b54b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:11:28.427486 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.427454    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-podinfo" pod "9e84b77a-82d5-4863-82c8-dee9669b54b2" (UID: "9e84b77a-82d5-4863-82c8-dee9669b54b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:11:28.427729 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.427626    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-envoy" pod "9e84b77a-82d5-4863-82c8-dee9669b54b2" (UID: "9e84b77a-82d5-4863-82c8-dee9669b54b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:28.528395 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.528363    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:28.528395 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.528393    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:28.528395 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.528405    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcpjx\" (UniqueName: \"kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-kube-api-access-qcpjx\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:28.528907 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.528416    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e84b77a-82d5-4863-82c8-dee9669b54b2-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:28.750739 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.750707    5066 generic.go:356] "Generic (PLEG): container finished" podID="9e84b77a-82d5-4863-82c8-dee9669b54b2" containerID="a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32" exitCode=0
Oct 30 06:11:28.750929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.750757    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"
Oct 30 06:11:28.750929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.750792    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" event={"ID":"9e84b77a-82d5-4863-82c8-dee9669b54b2","Type":"ContainerDied","Data":"a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32"}
Oct 30 06:11:28.750929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.750854    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q" event={"ID":"9e84b77a-82d5-4863-82c8-dee9669b54b2","Type":"ContainerDied","Data":"02efca8edb4b91b9ce5a462c5124d1baf3b4733d64d80e08653da59e4eb98f33"}
Oct 30 06:11:28.750929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.750883    5066 scope.go:122] "RemoveContainer" containerID="a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32"
Oct 30 06:11:28.763395 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.763367    5066 scope.go:122] "RemoveContainer" containerID="a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32"
Oct 30 06:11:28.763718 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:11:28.763690    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32\": container with ID starting with a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32 not found: ID does not exist" containerID="a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32"
Oct 30 06:11:28.763796 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.763729    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32"} err="failed to get container status \"a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32\": rpc error: code = NotFound desc = could not find container \"a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32\": container with ID starting with a5b1fd9354be3baa8699d55ceb563143e7dcfb2e97db4b2c47c998a3bb265d32 not found: ID does not exist"
Oct 30 06:11:28.812697 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.812663    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"]
Oct 30 06:11:28.824627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:28.824555    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-0d64c9-openshift-default-585f89bc6f-9q94q"]
Oct 30 06:11:29.534543 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:29.534509    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:30.403815 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:30.403784    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e84b77a-82d5-4863-82c8-dee9669b54b2" path="/var/lib/kubelet/pods/9e84b77a-82d5-4863-82c8-dee9669b54b2/volumes"
Oct 30 06:11:31.321230 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:31.321194    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:31.500117 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:31.500082    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:31.679993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:31.679956    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:32.133473 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:32.133335    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:32.305497 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:32.305456    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:32.553569 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:11:32.553539    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:11:33.061859 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:33.061822    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:33.288380 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:33.288341    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:34.827775 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:34.827738    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:35.471656 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:35.471623    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"]
Oct 30 06:11:35.558980 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:35.558942    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:36.465379 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:36.465343    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:36.783137 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:36.783052    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:11:36.789244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:36.789215    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:11:37.261402 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:37.261370    5066 scope.go:122] "RemoveContainer" containerID="001c087aeb50830b1a330f31bd7e2d0dc6a22c830455384d410876be1f1ea016"
Oct 30 06:11:38.171160 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:38.171122    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:38.330362 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:38.330321    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:38.488185 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:38.488080    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:39.669156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:39.669120    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"]
Oct 30 06:11:39.765808 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:39.765746    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:41.833599 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:41.833555    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:43.318149 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:43.318114    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:44.817557 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:44.817523    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:46.517928 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:46.517878    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:48.131632 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:48.131601    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:49.916430 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:49.916392    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:50.507639 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.507562    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll" podUID="a3c8e518-7e2a-4f99-8158-a7eb6932b2aa" containerName="main" containerID="cri-o://198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8" gracePeriod=45
Oct 30 06:11:50.851340 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.851315    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:11:50.949416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.949385    5066 generic.go:356] "Generic (PLEG): container finished" podID="a3c8e518-7e2a-4f99-8158-a7eb6932b2aa" containerID="198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8" exitCode=0
Oct 30 06:11:50.949416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.949427    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"
Oct 30 06:11:50.949936 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.949457    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll" event={"ID":"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa","Type":"ContainerDied","Data":"198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8"}
Oct 30 06:11:50.949936 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.949502    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll" event={"ID":"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa","Type":"ContainerDied","Data":"6bc372c60a8690609bf963cdb5201b4389ec88a620c04f561fb61f672192ac5e"}
Oct 30 06:11:50.949936 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.949520    5066 scope.go:122] "RemoveContainer" containerID="198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8"
Oct 30 06:11:50.962419 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.962396    5066 scope.go:122] "RemoveContainer" containerID="198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8"
Oct 30 06:11:50.962755 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:11:50.962729    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8\": container with ID starting with 198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8 not found: ID does not exist" containerID="198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8"
Oct 30 06:11:50.962852 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.962767    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8"} err="failed to get container status \"198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8\": rpc error: code = NotFound desc = could not find container \"198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8\": container with ID starting with 198bafdd35e96ebf9495a666328519f3dcd339264c4a74ea7ea2fb812d5d29a8 not found: ID does not exist"
Oct 30 06:11:50.981757 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.981731    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-dshm\") pod \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") "
Oct 30 06:11:50.981922 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.981772    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tmp-dir\") pod \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") "
Oct 30 06:11:50.981922 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.981827    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-kube-api-access-4zzjw\" (UniqueName: \"kubernetes.io/projected/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-kube-api-access-4zzjw\") pod \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") "
Oct 30 06:11:50.981922 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.981910    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-home\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-home\") pod \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") "
Oct 30 06:11:50.982112 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.981999    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tls-certs\") pod \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") "
Oct 30 06:11:50.982112 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.982100    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-model-cache\") pod \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\" (UID: \"a3c8e518-7e2a-4f99-8158-a7eb6932b2aa\") "
Oct 30 06:11:50.982234 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.982106    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tmp-dir" pod "a3c8e518-7e2a-4f99-8158-a7eb6932b2aa" (UID: "a3c8e518-7e2a-4f99-8158-a7eb6932b2aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:50.982312 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.982281    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-home" pod "a3c8e518-7e2a-4f99-8158-a7eb6932b2aa" (UID: "a3c8e518-7e2a-4f99-8158-a7eb6932b2aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:50.982478 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.982452    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-model-cache" pod "a3c8e518-7e2a-4f99-8158-a7eb6932b2aa" (UID: "a3c8e518-7e2a-4f99-8158-a7eb6932b2aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:50.982816 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.982793    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:50.982934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.982820    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:50.982934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.982858    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:50.984534 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.984514    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-kube-api-access-4zzjw" pod "a3c8e518-7e2a-4f99-8158-a7eb6932b2aa" (UID: "a3c8e518-7e2a-4f99-8158-a7eb6932b2aa"). InnerVolumeSpecName "kube-api-access-4zzjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:11:50.984727 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.984705    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tls-certs" pod "a3c8e518-7e2a-4f99-8158-a7eb6932b2aa" (UID: "a3c8e518-7e2a-4f99-8158-a7eb6932b2aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:11:50.984979 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:50.984965    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-dshm" pod "a3c8e518-7e2a-4f99-8158-a7eb6932b2aa" (UID: "a3c8e518-7e2a-4f99-8158-a7eb6932b2aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:51.084917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:51.084815    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:51.084917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:51.084852    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:51.084917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:51.084865    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zzjw\" (UniqueName: \"kubernetes.io/projected/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa-kube-api-access-4zzjw\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:51.303495 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:51.303459    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"]
Oct 30 06:11:51.315574 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:51.315536    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w0-cfe14f/auth-test-model-cfe14f-kserve-78cc66fd89-wzwll"]
Oct 30 06:11:51.627786 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:51.627746    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:52.403252 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:52.403214    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3c8e518-7e2a-4f99-8158-a7eb6932b2aa" path="/var/lib/kubelet/pods/a3c8e518-7e2a-4f99-8158-a7eb6932b2aa/volumes"
Oct 30 06:11:53.357963 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:53.357930    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:54.692486 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:54.692448    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9" podUID="f718b500-3f4c-499a-b8a6-a6001e27e7f3" containerName="main" containerID="cri-o://539848ddfb5516368856c881bf6dc79b6404673a31dff21ac33626bb6a479878" gracePeriod=45
Oct 30 06:11:54.857369 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:54.856312    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:54.980436 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:54.980406    5066 generic.go:356] "Generic (PLEG): container finished" podID="f718b500-3f4c-499a-b8a6-a6001e27e7f3" containerID="539848ddfb5516368856c881bf6dc79b6404673a31dff21ac33626bb6a479878" exitCode=0
Oct 30 06:11:54.980563 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:54.980483    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9" event={"ID":"f718b500-3f4c-499a-b8a6-a6001e27e7f3","Type":"ContainerDied","Data":"539848ddfb5516368856c881bf6dc79b6404673a31dff21ac33626bb6a479878"}
Oct 30 06:11:55.010137 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.010112    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:11:55.142472 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.142437    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tmp-dir\") pod \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") "
Oct 30 06:11:55.142645 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.142541    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-model-cache\") pod \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") "
Oct 30 06:11:55.142645 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.142573    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tls-certs\" (UniqueName: \"kubernetes.io/secret/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tls-certs\") pod \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") "
Oct 30 06:11:55.142790 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.142669    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f718b500-3f4c-499a-b8a6-a6001e27e7f3-kube-api-access-ts7rw\" (UniqueName: \"kubernetes.io/projected/f718b500-3f4c-499a-b8a6-a6001e27e7f3-kube-api-access-ts7rw\") pod \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") "
Oct 30 06:11:55.142790 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.142701    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tmp-dir" pod "f718b500-3f4c-499a-b8a6-a6001e27e7f3" (UID: "f718b500-3f4c-499a-b8a6-a6001e27e7f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:55.142790 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.142779    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-home\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-home\") pod \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") "
Oct 30 06:11:55.142943 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.142837    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-model-cache" pod "f718b500-3f4c-499a-b8a6-a6001e27e7f3" (UID: "f718b500-3f4c-499a-b8a6-a6001e27e7f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:55.142943 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.142891    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-dshm\") pod \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\" (UID: \"f718b500-3f4c-499a-b8a6-a6001e27e7f3\") "
Oct 30 06:11:55.143045 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.142967    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-home" pod "f718b500-3f4c-499a-b8a6-a6001e27e7f3" (UID: "f718b500-3f4c-499a-b8a6-a6001e27e7f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:55.143465 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.143444    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:55.143576 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.143467    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:55.143576 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.143479    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:55.145150 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.145128    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-dshm" pod "f718b500-3f4c-499a-b8a6-a6001e27e7f3" (UID: "f718b500-3f4c-499a-b8a6-a6001e27e7f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:11:55.145551 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.145531    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tls-certs" pod "f718b500-3f4c-499a-b8a6-a6001e27e7f3" (UID: "f718b500-3f4c-499a-b8a6-a6001e27e7f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:11:55.145551 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.145546    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f718b500-3f4c-499a-b8a6-a6001e27e7f3-kube-api-access-ts7rw" pod "f718b500-3f4c-499a-b8a6-a6001e27e7f3" (UID: "f718b500-3f4c-499a-b8a6-a6001e27e7f3"). InnerVolumeSpecName "kube-api-access-ts7rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:11:55.244672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.244561    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts7rw\" (UniqueName: \"kubernetes.io/projected/f718b500-3f4c-499a-b8a6-a6001e27e7f3-kube-api-access-ts7rw\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:55.244672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.244607    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f718b500-3f4c-499a-b8a6-a6001e27e7f3-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:55.244672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.244623    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f718b500-3f4c-499a-b8a6-a6001e27e7f3-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:11:55.771813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.771769    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"]
Oct 30 06:11:55.772394 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.772375    5066 cpu_manager.go:414] "containerMap: removing container" podUID="f718b500-3f4c-499a-b8a6-a6001e27e7f3" containerName="main"
Oct 30 06:11:55.772485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.772396    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f718b500-3f4c-499a-b8a6-a6001e27e7f3" containerName="main"
Oct 30 06:11:55.772485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.772415    5066 cpu_manager.go:414] "containerMap: removing container" podUID="9e84b77a-82d5-4863-82c8-dee9669b54b2" containerName="istio-proxy"
Oct 30 06:11:55.772485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.772420    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e84b77a-82d5-4863-82c8-dee9669b54b2" containerName="istio-proxy"
Oct 30 06:11:55.772485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.772458    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a3c8e518-7e2a-4f99-8158-a7eb6932b2aa" containerName="main"
Oct 30 06:11:55.772485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.772463    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3c8e518-7e2a-4f99-8158-a7eb6932b2aa" containerName="main"
Oct 30 06:11:55.772485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.772472    5066 cpu_manager.go:414] "containerMap: removing container" podUID="1a325b74-cd9e-4678-a445-56cb549a12fb" containerName="istio-proxy"
Oct 30 06:11:55.772485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.772477    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a325b74-cd9e-4678-a445-56cb549a12fb" containerName="istio-proxy"
Oct 30 06:11:55.772763 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.772606    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="f718b500-3f4c-499a-b8a6-a6001e27e7f3" containerName="main"
Oct 30 06:11:55.772763 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.772617    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e84b77a-82d5-4863-82c8-dee9669b54b2" containerName="istio-proxy"
Oct 30 06:11:55.772763 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.772631    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3c8e518-7e2a-4f99-8158-a7eb6932b2aa" containerName="main"
Oct 30 06:11:55.772763 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.772637    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a325b74-cd9e-4678-a445-56cb549a12fb" containerName="istio-proxy"
Oct 30 06:11:55.776558 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.776485    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:55.804144 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.803897    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"]
Oct 30 06:11:55.806677 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.806654    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w6-477f6b\"/\"openshift-service-ca.crt\""
Oct 30 06:11:55.806971 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.806954    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w6-477f6b\"/\"kube-root-ca.crt\""
Oct 30 06:11:55.807638 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.807619    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w6-477f6b\"/\"auth-test-model-477f6b-kserve-self-signed-certs\""
Oct 30 06:11:55.807779 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.807698    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w6-477f6b\"/\"default-dockercfg-5t7qq\""
Oct 30 06:11:55.898391 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.898353    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:55.955436 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.955399    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkzlw\" (UniqueName: \"kubernetes.io/projected/a663edf9-fdf4-4520-bc00-96ed2c535c5c-kube-api-access-hkzlw\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:55.955684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.955454    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-model-cache\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:55.955684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.955626    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tls-certs\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:55.955684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.955673    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-dshm\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:55.955874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.955815    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-home\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:55.955993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.955974    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tmp-dir\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:55.990363 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.990327    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"
Oct 30 06:11:55.990363 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.990351    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9" event={"ID":"f718b500-3f4c-499a-b8a6-a6001e27e7f3","Type":"ContainerDied","Data":"83e0addf443e00747c0e5b50eea97750fba1f62e9f9e5d3b6466d98362dc590a"}
Oct 30 06:11:55.990647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:55.990403    5066 scope.go:122] "RemoveContainer" containerID="539848ddfb5516368856c881bf6dc79b6404673a31dff21ac33626bb6a479878"
Oct 30 06:11:56.064784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.060924    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tmp-dir\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.064784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.061012    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkzlw\" (UniqueName: \"kubernetes.io/projected/a663edf9-fdf4-4520-bc00-96ed2c535c5c-kube-api-access-hkzlw\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.064784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.061050    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-model-cache\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.064784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.061164    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tls-certs\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.064784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.061224    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-dshm\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.064784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.061311    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-home\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.068562 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.068524    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"]
Oct 30 06:11:56.069217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.069189    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-model-cache\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.069386 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.069212    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-home\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.074016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.073984    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tmp-dir\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.093739 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.093712    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkzlw\" (UniqueName: \"kubernetes.io/projected/a663edf9-fdf4-4520-bc00-96ed2c535c5c-kube-api-access-hkzlw\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.111711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.111677    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d3e7e6/test-model-d3e7e6-kserve-c5945747-g2tr9"]
Oct 30 06:11:56.136663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.131973    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-dshm\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.139108 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.139037    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tls-certs\") pod \"auth-test-model-477f6b-kserve-6b4d855665-8xth6\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") " pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.403800 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.403768    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f718b500-3f4c-499a-b8a6-a6001e27e7f3" path="/var/lib/kubelet/pods/f718b500-3f4c-499a-b8a6-a6001e27e7f3/volumes"
Oct 30 06:11:56.410283 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.410261    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:56.518610 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.518276    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:56.725053 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:56.725023    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"]
Oct 30 06:11:56.725181 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:11:56.725064    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda663edf9_fdf4_4520_bc00_96ed2c535c5c.slice/crio-0ac09f9a58a01d61880a64b641388509d0ca72261119dea89fb962da2ef9853a WatchSource:0}: Error finding container 0ac09f9a58a01d61880a64b641388509d0ca72261119dea89fb962da2ef9853a: Status 404 returned error can't find the container with id 0ac09f9a58a01d61880a64b641388509d0ca72261119dea89fb962da2ef9853a
Oct 30 06:11:57.000334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:57.000244    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6" event={"ID":"a663edf9-fdf4-4520-bc00-96ed2c535c5c","Type":"ContainerStarted","Data":"aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1"}
Oct 30 06:11:57.000334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:57.000286    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6" event={"ID":"a663edf9-fdf4-4520-bc00-96ed2c535c5c","Type":"ContainerStarted","Data":"0ac09f9a58a01d61880a64b641388509d0ca72261119dea89fb962da2ef9853a"}
Oct 30 06:11:57.410712 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:57.410669    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:11:58.326325 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:58.326280    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:11:59.832005 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:11:59.831959    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:01.416300 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:01.416265    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:02.553746 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:12:02.553712    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:12:03.044545 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:03.044439    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:04.613770 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:04.613741    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:05.941229 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:05.941179    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6" podStartSLOduration=10.941165118 podStartE2EDuration="10.941165118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:11:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:11:57.08687133 +0000 UTC m=+2420.832156663" watchObservedRunningTime="2026-10-30 06:12:05.941165118 +0000 UTC m=+2429.686450449"
Oct 30 06:12:05.942462 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:05.942438    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"]
Oct 30 06:12:05.946741 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:05.946722    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:05.968766 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:05.968739    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-0f980e-serving-cert\""
Oct 30 06:12:05.975020 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:05.974983    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"]
Oct 30 06:12:06.003717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.003686    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p45zk\" (UniqueName: \"kubernetes.io/projected/3289c892-df28-4817-aef4-2291dc0d3a4b-kube-api-access-p45zk\") pod \"maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.003934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.003758    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.004070 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.004052    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3289c892-df28-4817-aef4-2291dc0d3a4b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.004188 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.004165    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.105627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.105559    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3289c892-df28-4817-aef4-2291dc0d3a4b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.105805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.105657    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.105805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.105764    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p45zk\" (UniqueName: \"kubernetes.io/projected/3289c892-df28-4817-aef4-2291dc0d3a4b-kube-api-access-p45zk\") pod \"maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.105895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.105833    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.106303 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.106279    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3289c892-df28-4817-aef4-2291dc0d3a4b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.108518 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.108480    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.108518 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.108518    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.167409 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.167361    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:06.168724 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.168699    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p45zk\" (UniqueName: \"kubernetes.io/projected/3289c892-df28-4817-aef4-2291dc0d3a4b-kube-api-access-p45zk\") pod \"maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.280375 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.280285    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:06.329199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.329165    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:06.410388 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.410354    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:12:06.423788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.423761    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:12:06.607866 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.607840    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"]
Oct 30 06:12:06.608703 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:12:06.608670    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3289c892_df28_4817_aef4_2291dc0d3a4b.slice/crio-d09be7215b2c8ce1f836c9aad8d617ef5e96be432e2dd9bc8e5ccd6bb1745f82 WatchSource:0}: Error finding container d09be7215b2c8ce1f836c9aad8d617ef5e96be432e2dd9bc8e5ccd6bb1745f82: Status 404 returned error can't find the container with id d09be7215b2c8ce1f836c9aad8d617ef5e96be432e2dd9bc8e5ccd6bb1745f82
Oct 30 06:12:06.610119 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.610099    5066 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 06:12:06.688527 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:06.688471    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:07.080073 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:07.079996    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2" event={"ID":"3289c892-df28-4817-aef4-2291dc0d3a4b","Type":"ContainerStarted","Data":"d09be7215b2c8ce1f836c9aad8d617ef5e96be432e2dd9bc8e5ccd6bb1745f82"}
Oct 30 06:12:07.129445 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:07.129153    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:12:07.217961 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:07.217925    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:07.414682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:07.407052    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:08.025609 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:08.025022    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:08.090864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:08.090817    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2" event={"ID":"3289c892-df28-4817-aef4-2291dc0d3a4b","Type":"ContainerStarted","Data":"93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568"}
Oct 30 06:12:08.091361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:08.090982    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:08.176690 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:08.176630    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2" podStartSLOduration=2.779806025 podStartE2EDuration="3.176611164s" totalImagesPullingTime="396.805139ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:12:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:12:08.1719559 +0000 UTC m=+2431.917241257" watchObservedRunningTime="2026-10-30 06:12:08.176611164 +0000 UTC m=+2431.921896496"
Oct 30 06:12:08.776945 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:08.776911    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:10.006562 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:10.006526    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:11.595555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:11.595499    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:11.777848 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:11.777811    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:13.161757 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:13.161722    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:13.365760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:13.365730    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:13.590984 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:13.590883    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:13.887440 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:13.887279    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:14.109394 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:14.109362    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:12:15.674276 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:15.674237    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:17.568553 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.568521    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"]
Oct 30 06:12:17.573535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.573512    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.598048 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.598014    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"]
Oct 30 06:12:17.606493 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.606462    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c73fcb\"/\"openshift-service-ca.crt\""
Oct 30 06:12:17.606677 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.606545    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c73fcb\"/\"kube-root-ca.crt\""
Oct 30 06:12:17.608683 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.608657    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c73fcb\"/\"auth-test-model-c73fcb-kserve-self-signed-certs\""
Oct 30 06:12:17.608821 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.608693    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c73fcb\"/\"default-dockercfg-dm7qh\""
Oct 30 06:12:17.669924 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.669889    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-dshm\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.670085 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.669943    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27szc\" (UniqueName: \"kubernetes.io/projected/7c9c9518-ad3f-478c-964e-2987da51aaa7-kube-api-access-27szc\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.670139 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.670119    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-model-cache\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.670183 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.670159    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-tmp-dir\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.670278 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.670264    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c9c9518-ad3f-478c-964e-2987da51aaa7-tls-certs\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.670345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.670328    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-home\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.672249 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.672222    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:17.771722 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.771687    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c9c9518-ad3f-478c-964e-2987da51aaa7-tls-certs\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.771923 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.771796    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-home\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.771923 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.771873    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-dshm\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.772047 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.771920    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27szc\" (UniqueName: \"kubernetes.io/projected/7c9c9518-ad3f-478c-964e-2987da51aaa7-kube-api-access-27szc\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.772105 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.772043    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-model-cache\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.772105 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.772086    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-tmp-dir\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.772538 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.772514    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-tmp-dir\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.773198 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.773171    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-model-cache\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.773300 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.773242    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-home\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.775471 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.775450    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-dshm\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.780190 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.780167    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c9c9518-ad3f-478c-964e-2987da51aaa7-tls-certs\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.800399 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.800366    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27szc\" (UniqueName: \"kubernetes.io/projected/7c9c9518-ad3f-478c-964e-2987da51aaa7-kube-api-access-27szc\") pod \"auth-test-model-c73fcb-kserve-c98776c57-twr2b\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") " pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:17.846255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.846170    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:17.897156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:17.895405    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:18.208660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:18.208630    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:18.276838 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:18.276804    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"]
Oct 30 06:12:18.277010 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:12:18.276991    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c9c9518_ad3f_478c_964e_2987da51aaa7.slice/crio-bb7a0428214c65b2721e66615c9f64581438397d9f4caed08b8506e117cc24d9 WatchSource:0}: Error finding container bb7a0428214c65b2721e66615c9f64581438397d9f4caed08b8506e117cc24d9: Status 404 returned error can't find the container with id bb7a0428214c65b2721e66615c9f64581438397d9f4caed08b8506e117cc24d9
Oct 30 06:12:19.211267 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:19.211227    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b" event={"ID":"7c9c9518-ad3f-478c-964e-2987da51aaa7","Type":"ContainerStarted","Data":"2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868"}
Oct 30 06:12:19.211267 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:19.211269    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b" event={"ID":"7c9c9518-ad3f-478c-964e-2987da51aaa7","Type":"ContainerStarted","Data":"bb7a0428214c65b2721e66615c9f64581438397d9f4caed08b8506e117cc24d9"}
Oct 30 06:12:19.259459 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:19.259410    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b" podStartSLOduration=2.259395837 podStartE2EDuration="2.259395837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:12:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:12:19.257570491 +0000 UTC m=+2443.002855822" watchObservedRunningTime="2026-10-30 06:12:19.259395837 +0000 UTC m=+2443.004681169"
Oct 30 06:12:19.450613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:19.448247    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:19.898163 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:19.898119    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:21.127074 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:21.127006    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:22.715145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:22.715104    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:24.488766 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:24.488732    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:24.649692 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:24.649655    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:26.122574 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:26.122535    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:27.681835 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:27.681781    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:27.897964 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:27.897927    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:27.910943 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:27.910917    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:28.286834 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:28.286807    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:12:29.315105 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:29.315070    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:29.486906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:29.486859    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:31.120758 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:31.120724    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:32.256279 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:32.256250    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:32.552850 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:12:32.552768    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:12:32.725158 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:32.724892    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:33.183757 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:33.183698    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:34.218949 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:34.218913    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:35.864601 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:35.863486    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:37.018316 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.018277    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"]
Oct 30 06:12:37.019212 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.019156    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8" podUID="f5e2d33b-da00-43f1-8191-2a8f47eadf8c" containerName="maas-api" containerID="cri-o://4c5e9c25a3e8292d766a9498d013a837319e43066bf0db6f8244ab29f6f95994" gracePeriod=30
Oct 30 06:12:37.139278 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.139236    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:37.430240 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.423968    5066 generic.go:356] "Generic (PLEG): container finished" podID="f5e2d33b-da00-43f1-8191-2a8f47eadf8c" containerID="4c5e9c25a3e8292d766a9498d013a837319e43066bf0db6f8244ab29f6f95994" exitCode=0
Oct 30 06:12:37.436009 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.433668    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8" event={"ID":"f5e2d33b-da00-43f1-8191-2a8f47eadf8c","Type":"ContainerDied","Data":"4c5e9c25a3e8292d766a9498d013a837319e43066bf0db6f8244ab29f6f95994"}
Oct 30 06:12:37.436009 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.435811    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:12:37.486642 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.485440    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-kube-api-access-zqsm9\" (UniqueName: \"kubernetes.io/projected/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-kube-api-access-zqsm9\") pod \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") "
Oct 30 06:12:37.486642 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.485536    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-metrics-certs\") pod \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") "
Oct 30 06:12:37.486642 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.485611    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-openshift-service-ca\") pod \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") "
Oct 30 06:12:37.486642 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.485794    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-maas-api-tls\") pod \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\" (UID: \"f5e2d33b-da00-43f1-8191-2a8f47eadf8c\") "
Oct 30 06:12:37.492760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.492433    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-metrics-certs" pod "f5e2d33b-da00-43f1-8191-2a8f47eadf8c" (UID: "f5e2d33b-da00-43f1-8191-2a8f47eadf8c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:12:37.506613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.500475    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-maas-api-tls" pod "f5e2d33b-da00-43f1-8191-2a8f47eadf8c" (UID: "f5e2d33b-da00-43f1-8191-2a8f47eadf8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:12:37.506613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.501108    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-openshift-service-ca" pod "f5e2d33b-da00-43f1-8191-2a8f47eadf8c" (UID: "f5e2d33b-da00-43f1-8191-2a8f47eadf8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:12:37.509922 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.509694    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-kube-api-access-zqsm9" pod "f5e2d33b-da00-43f1-8191-2a8f47eadf8c" (UID: "f5e2d33b-da00-43f1-8191-2a8f47eadf8c"). InnerVolumeSpecName "kube-api-access-zqsm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:12:37.591805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.591709    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:37.591805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.591747    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqsm9\" (UniqueName: \"kubernetes.io/projected/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-kube-api-access-zqsm9\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:37.591805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.591758    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:37.591805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.591766    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e2d33b-da00-43f1-8191-2a8f47eadf8c-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:37.685979 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.685942    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:37.875143 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:37.875051    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:38.044619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:38.044569    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:38.432358 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:38.432296    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8" event={"ID":"f5e2d33b-da00-43f1-8191-2a8f47eadf8c","Type":"ContainerDied","Data":"8014ff346efdae2500f943f3ce10209e060c5d9dd37c37bde011161470cde11d"}
Oct 30 06:12:38.432517 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:38.432363    5066 scope.go:122] "RemoveContainer" containerID="4c5e9c25a3e8292d766a9498d013a837319e43066bf0db6f8244ab29f6f95994"
Oct 30 06:12:38.432517 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:38.432309    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"
Oct 30 06:12:38.521060 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:38.521022    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:38.521692 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:38.521567    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"]
Oct 30 06:12:38.534340 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:38.534313    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d3e7e6-7cb9dcd64c-lmdh8"]
Oct 30 06:12:39.939350 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:39.939316    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:40.465611 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:40.464707    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5e2d33b-da00-43f1-8191-2a8f47eadf8c" path="/var/lib/kubelet/pods/f5e2d33b-da00-43f1-8191-2a8f47eadf8c/volumes"
Oct 30 06:12:41.732782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:41.732743    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:43.435544 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:43.435481    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:45.086516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:45.086482    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:45.265956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:45.265917    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:45.363481 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:45.363397    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"]
Oct 30 06:12:45.363709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:45.363627    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" podUID="b307c327-0d78-4749-99e4-30471e9410c0" containerName="istio-proxy" containerID="cri-o://75a17c44e1eaddfd91d1f0464e91ac21892e0d01699c02016adef8a28d09072f" gracePeriod=30
Oct 30 06:12:45.446646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:45.446570    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:46.318780 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:46.318740    5066 patch_prober.go:29] interesting pod/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:12:46.319193 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:46.318809    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" podUID="b307c327-0d78-4749-99e4-30471e9410c0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:12:47.020670 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:47.020637    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:48.128339 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.128304    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:48.648478 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.648440    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"]
Oct 30 06:12:48.649190 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.649168    5066 cpu_manager.go:414] "containerMap: removing container" podUID="f5e2d33b-da00-43f1-8191-2a8f47eadf8c" containerName="maas-api"
Oct 30 06:12:48.649190 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.649190    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5e2d33b-da00-43f1-8191-2a8f47eadf8c" containerName="maas-api"
Oct 30 06:12:48.649410 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.649396    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5e2d33b-da00-43f1-8191-2a8f47eadf8c" containerName="maas-api"
Oct 30 06:12:48.653158 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.653139    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.680390 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.680356    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-db0540-serving-cert\""
Oct 30 06:12:48.685779 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.685751    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"]
Oct 30 06:12:48.813193 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.813161    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-metrics-certs\") pod \"maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.813368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.813335    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.813495 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.813481    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.813533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.813508    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7vt2\" (UniqueName: \"kubernetes.io/projected/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-kube-api-access-j7vt2\") pod \"maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.821688 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.821663    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:48.928723 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.928686    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-metrics-certs\") pod \"maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.931375 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.931333    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.933665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.933629    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.933985 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.933965    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7vt2\" (UniqueName: \"kubernetes.io/projected/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-kube-api-access-j7vt2\") pod \"maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.934242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.933388    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.938824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.938659    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.939790 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.939766    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-metrics-certs\") pod \"maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.964462 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.964413    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7vt2\" (UniqueName: \"kubernetes.io/projected/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-kube-api-access-j7vt2\") pod \"maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:48.967607 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:48.967557    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:49.029455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:49.029299    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:49.294102 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:49.294074    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"]
Oct 30 06:12:49.298215 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:12:49.298183    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d4c399_5239_4ac4_b4f8_a15ecf2313fc.slice/crio-2eddd2c49c34624487f502f9af7b8b68a733479fb8489506aeaf9e8cf2f9dd8e WatchSource:0}: Error finding container 2eddd2c49c34624487f502f9af7b8b68a733479fb8489506aeaf9e8cf2f9dd8e: Status 404 returned error can't find the container with id 2eddd2c49c34624487f502f9af7b8b68a733479fb8489506aeaf9e8cf2f9dd8e
Oct 30 06:12:49.584893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:49.584793    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8" event={"ID":"44d4c399-5239-4ac4-b4f8-a15ecf2313fc","Type":"ContainerStarted","Data":"2eddd2c49c34624487f502f9af7b8b68a733479fb8489506aeaf9e8cf2f9dd8e"}
Oct 30 06:12:50.598876 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.598769    5066 generic.go:356] "Generic (PLEG): container finished" podID="b307c327-0d78-4749-99e4-30471e9410c0" containerID="75a17c44e1eaddfd91d1f0464e91ac21892e0d01699c02016adef8a28d09072f" exitCode=0
Oct 30 06:12:50.598876 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.598803    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" event={"ID":"b307c327-0d78-4749-99e4-30471e9410c0","Type":"ContainerDied","Data":"75a17c44e1eaddfd91d1f0464e91ac21892e0d01699c02016adef8a28d09072f"}
Oct 30 06:12:50.601645 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.601571    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8" event={"ID":"44d4c399-5239-4ac4-b4f8-a15ecf2313fc","Type":"ContainerStarted","Data":"609d32e82d5220b155efb5e07722aaec44456552ac2e92b047958b30e0482eb3"}
Oct 30 06:12:50.602859 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.602834    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:50.613188 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.613161    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:50.685925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.685871    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8" podStartSLOduration=2.323462479 podStartE2EDuration="2.685853455s" totalImagesPullingTime="362.390976ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:12:50.684981017 +0000 UTC m=+2474.430266350" watchObservedRunningTime="2026-10-30 06:12:50.685853455 +0000 UTC m=+2474.431138788"
Oct 30 06:12:50.695779 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.695754    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:12:50.778171 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.778136    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-socket\") pod \"b307c327-0d78-4749-99e4-30471e9410c0\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") "
Oct 30 06:12:50.778440 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.778420    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-kube-api-access-7tl2w\" (UniqueName: \"kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-kube-api-access-7tl2w\") pod \"b307c327-0d78-4749-99e4-30471e9410c0\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") "
Oct 30 06:12:50.779247 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.779220    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-data\") pod \"b307c327-0d78-4749-99e4-30471e9410c0\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") "
Oct 30 06:12:50.779429 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.779408    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-credential-socket\") pod \"b307c327-0d78-4749-99e4-30471e9410c0\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") "
Oct 30 06:12:50.779678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.779655    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-istio-token\" (UniqueName: \"kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-istio-token\") pod \"b307c327-0d78-4749-99e4-30471e9410c0\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") "
Oct 30 06:12:50.779779 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.779741    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-envoy\") pod \"b307c327-0d78-4749-99e4-30471e9410c0\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") "
Oct 30 06:12:50.779848 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.779806    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b307c327-0d78-4749-99e4-30471e9410c0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b307c327-0d78-4749-99e4-30471e9410c0-istio-podinfo\") pod \"b307c327-0d78-4749-99e4-30471e9410c0\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") "
Oct 30 06:12:50.780935 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.780905    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-data" pod "b307c327-0d78-4749-99e4-30471e9410c0" (UID: "b307c327-0d78-4749-99e4-30471e9410c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:12:50.781146 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.781122    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-socket" pod "b307c327-0d78-4749-99e4-30471e9410c0" (UID: "b307c327-0d78-4749-99e4-30471e9410c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:12:50.782307 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.782285    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-credential-socket" pod "b307c327-0d78-4749-99e4-30471e9410c0" (UID: "b307c327-0d78-4749-99e4-30471e9410c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:12:50.803206 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.803024    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-istio-token" pod "b307c327-0d78-4749-99e4-30471e9410c0" (UID: "b307c327-0d78-4749-99e4-30471e9410c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:12:50.803206 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.803123    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b307c327-0d78-4749-99e4-30471e9410c0-istio-podinfo" pod "b307c327-0d78-4749-99e4-30471e9410c0" (UID: "b307c327-0d78-4749-99e4-30471e9410c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:12:50.803206 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.803178    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-kube-api-access-7tl2w" pod "b307c327-0d78-4749-99e4-30471e9410c0" (UID: "b307c327-0d78-4749-99e4-30471e9410c0"). InnerVolumeSpecName "kube-api-access-7tl2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:12:50.811058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.805003    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-envoy" pod "b307c327-0d78-4749-99e4-30471e9410c0" (UID: "b307c327-0d78-4749-99e4-30471e9410c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:12:50.853514 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.853478    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:50.884615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.883317    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-certs\") pod \"b307c327-0d78-4749-99e4-30471e9410c0\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") "
Oct 30 06:12:50.884615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.883400    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b307c327-0d78-4749-99e4-30471e9410c0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b307c327-0d78-4749-99e4-30471e9410c0-istiod-ca-cert\") pod \"b307c327-0d78-4749-99e4-30471e9410c0\" (UID: \"b307c327-0d78-4749-99e4-30471e9410c0\") "
Oct 30 06:12:50.884615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.884233    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:50.884615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.884257    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:50.884615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.884272    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:50.884615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.884285    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b307c327-0d78-4749-99e4-30471e9410c0-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:50.884615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.884297    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:50.884615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.884312    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tl2w\" (UniqueName: \"kubernetes.io/projected/b307c327-0d78-4749-99e4-30471e9410c0-kube-api-access-7tl2w\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:50.884615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.884326    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:50.903542 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.890869    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b307c327-0d78-4749-99e4-30471e9410c0-istiod-ca-cert" pod "b307c327-0d78-4749-99e4-30471e9410c0" (UID: "b307c327-0d78-4749-99e4-30471e9410c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:12:50.903542 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.891724    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-certs" pod "b307c327-0d78-4749-99e4-30471e9410c0" (UID: "b307c327-0d78-4749-99e4-30471e9410c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:12:50.999612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.999056    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b307c327-0d78-4749-99e4-30471e9410c0-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:50.999612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:50.999090    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b307c327-0d78-4749-99e4-30471e9410c0-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:12:51.610185 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:51.610157    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"
Oct 30 06:12:51.610667 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:51.610201    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8" event={"ID":"b307c327-0d78-4749-99e4-30471e9410c0","Type":"ContainerDied","Data":"8615c7ae68b6e152aa4522d7be5ded02e8580127ea25f09da144665e527332ac"}
Oct 30 06:12:51.610667 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:51.610266    5066 scope.go:122] "RemoveContainer" containerID="75a17c44e1eaddfd91d1f0464e91ac21892e0d01699c02016adef8a28d09072f"
Oct 30 06:12:51.684507 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:51.684473    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"]
Oct 30 06:12:51.698092 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:51.698058    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-d3e7e6-openshift-default-6c65f9d6ff-c9zh8"]
Oct 30 06:12:52.399720 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:52.399684    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:52.407852 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:52.407822    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b307c327-0d78-4749-99e4-30471e9410c0" path="/var/lib/kubelet/pods/b307c327-0d78-4749-99e4-30471e9410c0/volumes"
Oct 30 06:12:53.982921 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:53.982888    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:54.170616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:54.159782    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:55.812183 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:55.812140    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:56.472637 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:56.472580    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:56.640351 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:56.640318    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:57.318016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:57.317985    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:12:57.625409 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:57.625340    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:12:58.865887 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:12:58.865858    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:00.535681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:00.535639    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:02.040244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:02.040208    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:02.220618 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:02.220409    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:02.554815 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:13:02.554784    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:13:03.802445 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:03.802413    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:05.382174 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:05.382127    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:06.887134 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:06.887083    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:08.361706 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:08.360009    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:09.689820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:09.689789    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:09.902434 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:09.902406    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:11.415937 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:11.415896    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:12.963551 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:12.963503    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:14.495936 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:14.495899    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:15.973912 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:15.973882    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:17.498400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:17.498277    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:19.140164 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:19.140126    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:20.313558 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.313522    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"]
Oct 30 06:13:20.314403 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.313804    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29" podUID="ef623c5b-e3fb-4802-94ec-4dd0b445b441" containerName="maas-api" containerID="cri-o://2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1" gracePeriod=30
Oct 30 06:13:20.439167 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.439135    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:20.667947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.667920    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:13:20.745622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.742013    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-maas-api-tls\") pod \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") "
Oct 30 06:13:20.745622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.742810    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef623c5b-e3fb-4802-94ec-4dd0b445b441-kube-api-access-khq46\" (UniqueName: \"kubernetes.io/projected/ef623c5b-e3fb-4802-94ec-4dd0b445b441-kube-api-access-khq46\") pod \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") "
Oct 30 06:13:20.745622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.742914    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-metrics-certs\") pod \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") "
Oct 30 06:13:20.745622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.742994    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef623c5b-e3fb-4802-94ec-4dd0b445b441-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef623c5b-e3fb-4802-94ec-4dd0b445b441-openshift-service-ca\") pod \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\" (UID: \"ef623c5b-e3fb-4802-94ec-4dd0b445b441\") "
Oct 30 06:13:20.745622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.744111    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef623c5b-e3fb-4802-94ec-4dd0b445b441-openshift-service-ca" pod "ef623c5b-e3fb-4802-94ec-4dd0b445b441" (UID: "ef623c5b-e3fb-4802-94ec-4dd0b445b441"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:13:20.753389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.753351    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-maas-api-tls" pod "ef623c5b-e3fb-4802-94ec-4dd0b445b441" (UID: "ef623c5b-e3fb-4802-94ec-4dd0b445b441"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:13:20.757480 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.757448    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef623c5b-e3fb-4802-94ec-4dd0b445b441-kube-api-access-khq46" pod "ef623c5b-e3fb-4802-94ec-4dd0b445b441" (UID: "ef623c5b-e3fb-4802-94ec-4dd0b445b441"). InnerVolumeSpecName "kube-api-access-khq46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:13:20.757693 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.757638    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-metrics-certs" pod "ef623c5b-e3fb-4802-94ec-4dd0b445b441" (UID: "ef623c5b-e3fb-4802-94ec-4dd0b445b441"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:13:20.830482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.830445    5066 generic.go:356] "Generic (PLEG): container finished" podID="ef623c5b-e3fb-4802-94ec-4dd0b445b441" containerID="2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1" exitCode=0
Oct 30 06:13:20.830741 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.830717    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29" event={"ID":"ef623c5b-e3fb-4802-94ec-4dd0b445b441","Type":"ContainerDied","Data":"2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1"}
Oct 30 06:13:20.830827 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.830814    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29" event={"ID":"ef623c5b-e3fb-4802-94ec-4dd0b445b441","Type":"ContainerDied","Data":"9705fb6d307fc479560e6aead98574c7d406d44977bb33798acda5d986172a76"}
Oct 30 06:13:20.830864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.830849    5066 scope.go:122] "RemoveContainer" containerID="2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1"
Oct 30 06:13:20.831044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.831029    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"
Oct 30 06:13:20.845995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.845958    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khq46\" (UniqueName: \"kubernetes.io/projected/ef623c5b-e3fb-4802-94ec-4dd0b445b441-kube-api-access-khq46\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:13:20.846123 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.846000    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:13:20.846123 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.846016    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef623c5b-e3fb-4802-94ec-4dd0b445b441-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:13:20.846123 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.846031    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef623c5b-e3fb-4802-94ec-4dd0b445b441-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:13:20.849673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.849649    5066 scope.go:122] "RemoveContainer" containerID="2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1"
Oct 30 06:13:20.850078 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:13:20.850056    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1\": container with ID starting with 2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1 not found: ID does not exist" containerID="2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1"
Oct 30 06:13:20.850169 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.850089    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1"} err="failed to get container status \"2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1\": rpc error: code = NotFound desc = could not find container \"2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1\": container with ID starting with 2a8154040164f5b841f5d2a0a7903a0d3afa3ec57610dc83ef61831ca64ec3b1 not found: ID does not exist"
Oct 30 06:13:20.902672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.893152    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"]
Oct 30 06:13:20.924612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.914862    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0d0152-87db96b6b-cvr29"]
Oct 30 06:13:20.992820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:20.992783    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:21.163037 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:21.161981    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:21.650399 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:21.650316    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:22.403082 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:22.403050    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef623c5b-e3fb-4802-94ec-4dd0b445b441" path="/var/lib/kubelet/pods/ef623c5b-e3fb-4802-94ec-4dd0b445b441/volumes"
Oct 30 06:13:22.830801 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:22.830766    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:24.478992 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:24.478952    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:26.150529 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:26.150491    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:27.704269 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:27.704232    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:28.419706 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:28.419673    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:28.709004 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:28.708911    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:29.766690 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:29.766655    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:29.939606 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:29.939529    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:30.143358 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:30.143232    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:30.327150 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:30.327106    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:30.924545 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:30.924510    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:31.415566 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:31.415531    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:31.609459 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:31.609337    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:31.771828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:31.771726    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:32.384826 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:32.384793    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:32.556315 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:13:32.556278    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:13:33.306148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:33.306117    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"]
Oct 30 06:13:33.316688 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:33.316656    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:33.498640 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:33.498583    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:34.257634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.257605    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9210-73c70f"]
Oct 30 06:13:34.258225 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.258209    5066 cpu_manager.go:414] "containerMap: removing container" podUID="b307c327-0d78-4749-99e4-30471e9410c0" containerName="istio-proxy"
Oct 30 06:13:34.258225 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.258222    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b307c327-0d78-4749-99e4-30471e9410c0" containerName="istio-proxy"
Oct 30 06:13:34.258301 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.258265    5066 cpu_manager.go:414] "containerMap: removing container" podUID="ef623c5b-e3fb-4802-94ec-4dd0b445b441" containerName="maas-api"
Oct 30 06:13:34.258301 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.258270    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef623c5b-e3fb-4802-94ec-4dd0b445b441" containerName="maas-api"
Oct 30 06:13:34.258405 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.258392    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="b307c327-0d78-4749-99e4-30471e9410c0" containerName="istio-proxy"
Oct 30 06:13:34.258468 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.258417    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef623c5b-e3fb-4802-94ec-4dd0b445b441" containerName="maas-api"
Oct 30 06:13:34.262115 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.262088    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9210-73c70f"
Oct 30 06:13:34.275732 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.275706    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9210-73c70f"]
Oct 30 06:13:34.286721 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.286689    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4m56v\""
Oct 30 06:13:34.371380 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.371299    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8ncc\" (UniqueName: \"kubernetes.io/projected/db5ed49b-e6b2-46ab-90b6-86e417217d29-kube-api-access-s8ncc\") pod \"test-curl-9210-73c70f\" (UID: \"db5ed49b-e6b2-46ab-90b6-86e417217d29\") " pod="opendatahub/test-curl-9210-73c70f"
Oct 30 06:13:34.472902 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.472861    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8ncc\" (UniqueName: \"kubernetes.io/projected/db5ed49b-e6b2-46ab-90b6-86e417217d29-kube-api-access-s8ncc\") pod \"test-curl-9210-73c70f\" (UID: \"db5ed49b-e6b2-46ab-90b6-86e417217d29\") " pod="opendatahub/test-curl-9210-73c70f"
Oct 30 06:13:34.499606 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.499558    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8ncc\" (UniqueName: \"kubernetes.io/projected/db5ed49b-e6b2-46ab-90b6-86e417217d29-kube-api-access-s8ncc\") pod \"test-curl-9210-73c70f\" (UID: \"db5ed49b-e6b2-46ab-90b6-86e417217d29\") " pod="opendatahub/test-curl-9210-73c70f"
Oct 30 06:13:34.580895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.580806    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9210-73c70f"
Oct 30 06:13:34.817613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.815690    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:34.928558 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:34.928471    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9210-73c70f"]
Oct 30 06:13:34.945556 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:13:34.945516    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb5ed49b_e6b2_46ab_90b6_86e417217d29.slice/crio-8b650c00f0ab6e72199f3951b42032946823f97628a6ea0b6a4c4a4a83bba26b WatchSource:0}: Error finding container 8b650c00f0ab6e72199f3951b42032946823f97628a6ea0b6a4c4a4a83bba26b: Status 404 returned error can't find the container with id 8b650c00f0ab6e72199f3951b42032946823f97628a6ea0b6a4c4a4a83bba26b
Oct 30 06:13:35.511459 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:35.511430    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:35.723480 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:35.723444    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:35.961145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:35.961098    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9210-73c70f" event={"ID":"db5ed49b-e6b2-46ab-90b6-86e417217d29","Type":"ContainerStarted","Data":"7baea6525a321930e7866e332ffdcb12be141054550ce48c0f741e1012cd1200"}
Oct 30 06:13:35.961328 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:35.961163    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9210-73c70f" event={"ID":"db5ed49b-e6b2-46ab-90b6-86e417217d29","Type":"ContainerStarted","Data":"8b650c00f0ab6e72199f3951b42032946823f97628a6ea0b6a4c4a4a83bba26b"}
Oct 30 06:13:36.024166 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:36.024116    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9210-73c70f" podStartSLOduration=2.024101139 podStartE2EDuration="2.024101139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:13:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:13:36.02276498 +0000 UTC m=+2519.768050313" watchObservedRunningTime="2026-10-30 06:13:36.024101139 +0000 UTC m=+2519.769386471"
Oct 30 06:13:36.094264 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:36.094230    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:36.288269 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:36.287454    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:36.959029 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:36.958996    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9210-73c70f"]
Oct 30 06:13:36.963777 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:36.963749    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9210-73c70f"]
Oct 30 06:13:37.114771 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.114734    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9210-4e5884"]
Oct 30 06:13:37.115659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.115643    5066 cpu_manager.go:414] "containerMap: removing container" podUID="db5ed49b-e6b2-46ab-90b6-86e417217d29" containerName="test-curl-9210-73c70f"
Oct 30 06:13:37.115709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.115662    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db5ed49b-e6b2-46ab-90b6-86e417217d29" containerName="test-curl-9210-73c70f"
Oct 30 06:13:37.115851 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.115838    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="db5ed49b-e6b2-46ab-90b6-86e417217d29" containerName="test-curl-9210-73c70f"
Oct 30 06:13:37.120289 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.120270    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9210-4e5884"
Oct 30 06:13:37.135490 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.135457    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9210-4e5884"]
Oct 30 06:13:37.237480 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.237380    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj5zq\" (UniqueName: \"kubernetes.io/projected/0fe79012-35f8-4236-93ab-3da8fd9fa91d-kube-api-access-cj5zq\") pod \"test-curl-9210-4e5884\" (UID: \"0fe79012-35f8-4236-93ab-3da8fd9fa91d\") " pod="opendatahub/test-curl-9210-4e5884"
Oct 30 06:13:37.339090 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.339065    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj5zq\" (UniqueName: \"kubernetes.io/projected/0fe79012-35f8-4236-93ab-3da8fd9fa91d-kube-api-access-cj5zq\") pod \"test-curl-9210-4e5884\" (UID: \"0fe79012-35f8-4236-93ab-3da8fd9fa91d\") " pod="opendatahub/test-curl-9210-4e5884"
Oct 30 06:13:37.370962 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.370928    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj5zq\" (UniqueName: \"kubernetes.io/projected/0fe79012-35f8-4236-93ab-3da8fd9fa91d-kube-api-access-cj5zq\") pod \"test-curl-9210-4e5884\" (UID: \"0fe79012-35f8-4236-93ab-3da8fd9fa91d\") " pod="opendatahub/test-curl-9210-4e5884"
Oct 30 06:13:37.418537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.418507    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:37.439918 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.439888    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9210-4e5884"
Oct 30 06:13:37.600973 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.600941    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:37.773680 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.773550    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:37.806612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.803137    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9210-4e5884"]
Oct 30 06:13:37.985660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.985622    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9210-4e5884" event={"ID":"0fe79012-35f8-4236-93ab-3da8fd9fa91d","Type":"ContainerStarted","Data":"898e18c1d63c5e8cbe2aa4435e7d397926885364a392ac48c1e9fa653f385354"}
Oct 30 06:13:37.986210 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.985673    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9210-4e5884" event={"ID":"0fe79012-35f8-4236-93ab-3da8fd9fa91d","Type":"ContainerStarted","Data":"91d078426d5c6598c476ad94ea7ea988f74c58325662fed7e78e37a257c3efbd"}
Oct 30 06:13:37.986210 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:37.985757    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9210-73c70f" podUID="db5ed49b-e6b2-46ab-90b6-86e417217d29" containerName="test-curl-9210-73c70f" containerID="cri-o://7baea6525a321930e7866e332ffdcb12be141054550ce48c0f741e1012cd1200" gracePeriod=2
Oct 30 06:13:38.039679 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:38.035636    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9210-4e5884" podStartSLOduration=1.035617325 podStartE2EDuration="1.035617325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:13:38.030368479 +0000 UTC m=+2521.775653815" watchObservedRunningTime="2026-10-30 06:13:38.035617325 +0000 UTC m=+2521.780902649"
Oct 30 06:13:38.045822 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:13:38.045784    5066 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9210-73c70f\" is forbidden: User \"system:node:ip-10-0-136-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-173.ec2.internal' and this object" podUID="db5ed49b-e6b2-46ab-90b6-86e417217d29" pod="opendatahub/test-curl-9210-73c70f"
Oct 30 06:13:38.061735 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:13:38.061673    5066 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9210-73c70f\" is forbidden: User \"system:node:ip-10-0-136-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-173.ec2.internal' and this object" podUID="db5ed49b-e6b2-46ab-90b6-86e417217d29" pod="opendatahub/test-curl-9210-73c70f"
Oct 30 06:13:38.615648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:38.615615    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:38.926117 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:38.926075    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9210-4e5884"]
Oct 30 06:13:38.934974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:38.934943    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9210-4e5884"]
Oct 30 06:13:39.743162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:39.743070    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:40.002248 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:40.002151    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9210-4e5884" podUID="0fe79012-35f8-4236-93ab-3da8fd9fa91d" containerName="test-curl-9210-4e5884" containerID="cri-o://898e18c1d63c5e8cbe2aa4435e7d397926885364a392ac48c1e9fa653f385354" gracePeriod=2
Oct 30 06:13:40.017920 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:13:40.017884    5066 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9210-4e5884\" is forbidden: User \"system:node:ip-10-0-136-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-173.ec2.internal' and this object" podUID="0fe79012-35f8-4236-93ab-3da8fd9fa91d" pod="opendatahub/test-curl-9210-4e5884"
Oct 30 06:13:40.253904 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:40.253833    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9210-73c70f"
Oct 30 06:13:40.269353 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:13:40.269317    5066 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9210-4e5884\" is forbidden: User \"system:node:ip-10-0-136-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-173.ec2.internal' and this object" podUID="0fe79012-35f8-4236-93ab-3da8fd9fa91d" pod="opendatahub/test-curl-9210-4e5884"
Oct 30 06:13:40.293478 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:40.293436    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db5ed49b-e6b2-46ab-90b6-86e417217d29-kube-api-access-s8ncc\" (UniqueName: \"kubernetes.io/projected/db5ed49b-e6b2-46ab-90b6-86e417217d29-kube-api-access-s8ncc\") pod \"db5ed49b-e6b2-46ab-90b6-86e417217d29\" (UID: \"db5ed49b-e6b2-46ab-90b6-86e417217d29\") "
Oct 30 06:13:40.295816 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:40.295789    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db5ed49b-e6b2-46ab-90b6-86e417217d29-kube-api-access-s8ncc" pod "db5ed49b-e6b2-46ab-90b6-86e417217d29" (UID: "db5ed49b-e6b2-46ab-90b6-86e417217d29"). InnerVolumeSpecName "kube-api-access-s8ncc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:13:40.395008 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:40.394976    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8ncc\" (UniqueName: \"kubernetes.io/projected/db5ed49b-e6b2-46ab-90b6-86e417217d29-kube-api-access-s8ncc\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:13:40.403380 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:40.403350    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db5ed49b-e6b2-46ab-90b6-86e417217d29" path="/var/lib/kubelet/pods/db5ed49b-e6b2-46ab-90b6-86e417217d29/volumes"
Oct 30 06:13:40.680211 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:40.680178    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:40.848011 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:40.847984    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:41.010051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:41.009956    5066 generic.go:356] "Generic (PLEG): container finished" podID="db5ed49b-e6b2-46ab-90b6-86e417217d29" containerID="7baea6525a321930e7866e332ffdcb12be141054550ce48c0f741e1012cd1200" exitCode=137
Oct 30 06:13:41.010051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:41.010010    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9210-73c70f"
Oct 30 06:13:41.010289 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:41.010055    5066 scope.go:122] "RemoveContainer" containerID="7baea6525a321930e7866e332ffdcb12be141054550ce48c0f741e1012cd1200"
Oct 30 06:13:41.021743 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:41.021717    5066 scope.go:122] "RemoveContainer" containerID="7baea6525a321930e7866e332ffdcb12be141054550ce48c0f741e1012cd1200"
Oct 30 06:13:41.022264 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:13:41.022186    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7baea6525a321930e7866e332ffdcb12be141054550ce48c0f741e1012cd1200\": container with ID starting with 7baea6525a321930e7866e332ffdcb12be141054550ce48c0f741e1012cd1200 not found: ID does not exist" containerID="7baea6525a321930e7866e332ffdcb12be141054550ce48c0f741e1012cd1200"
Oct 30 06:13:41.022395 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:41.022310    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7baea6525a321930e7866e332ffdcb12be141054550ce48c0f741e1012cd1200"} err="failed to get container status \"7baea6525a321930e7866e332ffdcb12be141054550ce48c0f741e1012cd1200\": rpc error: code = NotFound desc = could not find container \"7baea6525a321930e7866e332ffdcb12be141054550ce48c0f741e1012cd1200\": container with ID starting with 7baea6525a321930e7866e332ffdcb12be141054550ce48c0f741e1012cd1200 not found: ID does not exist"
Oct 30 06:13:41.539728 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:41.539155    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:42.289061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:42.289027    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:42.299642 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:42.298520    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9210-4e5884"
Oct 30 06:13:42.324552 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:42.324514    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fe79012-35f8-4236-93ab-3da8fd9fa91d-kube-api-access-cj5zq\" (UniqueName: \"kubernetes.io/projected/0fe79012-35f8-4236-93ab-3da8fd9fa91d-kube-api-access-cj5zq\") pod \"0fe79012-35f8-4236-93ab-3da8fd9fa91d\" (UID: \"0fe79012-35f8-4236-93ab-3da8fd9fa91d\") "
Oct 30 06:13:42.337656 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:42.333193    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe79012-35f8-4236-93ab-3da8fd9fa91d-kube-api-access-cj5zq" pod "0fe79012-35f8-4236-93ab-3da8fd9fa91d" (UID: "0fe79012-35f8-4236-93ab-3da8fd9fa91d"). InnerVolumeSpecName "kube-api-access-cj5zq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:13:42.415402 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:42.415361    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0fe79012-35f8-4236-93ab-3da8fd9fa91d" path="/var/lib/kubelet/pods/0fe79012-35f8-4236-93ab-3da8fd9fa91d/volumes"
Oct 30 06:13:42.438654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:42.427258    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj5zq\" (UniqueName: \"kubernetes.io/projected/0fe79012-35f8-4236-93ab-3da8fd9fa91d-kube-api-access-cj5zq\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:13:42.474608 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:42.474485    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:43.031584 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:43.031552    5066 generic.go:356] "Generic (PLEG): container finished" podID="0fe79012-35f8-4236-93ab-3da8fd9fa91d" containerID="898e18c1d63c5e8cbe2aa4435e7d397926885364a392ac48c1e9fa653f385354" exitCode=137
Oct 30 06:13:43.031805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:43.031659    5066 scope.go:122] "RemoveContainer" containerID="898e18c1d63c5e8cbe2aa4435e7d397926885364a392ac48c1e9fa653f385354"
Oct 30 06:13:43.031805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:43.031675    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9210-4e5884"
Oct 30 06:13:43.045223 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:43.045189    5066 scope.go:122] "RemoveContainer" containerID="898e18c1d63c5e8cbe2aa4435e7d397926885364a392ac48c1e9fa653f385354"
Oct 30 06:13:43.045624 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:13:43.045580    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"898e18c1d63c5e8cbe2aa4435e7d397926885364a392ac48c1e9fa653f385354\": container with ID starting with 898e18c1d63c5e8cbe2aa4435e7d397926885364a392ac48c1e9fa653f385354 not found: ID does not exist" containerID="898e18c1d63c5e8cbe2aa4435e7d397926885364a392ac48c1e9fa653f385354"
Oct 30 06:13:43.045699 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:43.045638    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898e18c1d63c5e8cbe2aa4435e7d397926885364a392ac48c1e9fa653f385354"} err="failed to get container status \"898e18c1d63c5e8cbe2aa4435e7d397926885364a392ac48c1e9fa653f385354\": rpc error: code = NotFound desc = could not find container \"898e18c1d63c5e8cbe2aa4435e7d397926885364a392ac48c1e9fa653f385354\": container with ID starting with 898e18c1d63c5e8cbe2aa4435e7d397926885364a392ac48c1e9fa653f385354 not found: ID does not exist"
Oct 30 06:13:43.302054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:43.301968    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:45.209896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:45.209865    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:46.724433 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:46.724353    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:48.343648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.343603    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6" podUID="a663edf9-fdf4-4520-bc00-96ed2c535c5c" containerName="main" containerID="cri-o://aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1" gracePeriod=45
Oct 30 06:13:48.415460 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.415420    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:48.636671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.636640    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:13:48.719369 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.719302    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-dshm\") pod \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") "
Oct 30 06:13:48.719566 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.719480    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tmp-dir\") pod \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") "
Oct 30 06:13:48.719566 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.719546    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-home\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-home\") pod \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") "
Oct 30 06:13:48.719771 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.719633    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-model-cache\") pod \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") "
Oct 30 06:13:48.719771 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.719708    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tls-certs\" (UniqueName: \"kubernetes.io/secret/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tls-certs\") pod \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") "
Oct 30 06:13:48.719771 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.719748    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a663edf9-fdf4-4520-bc00-96ed2c535c5c-kube-api-access-hkzlw\" (UniqueName: \"kubernetes.io/projected/a663edf9-fdf4-4520-bc00-96ed2c535c5c-kube-api-access-hkzlw\") pod \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\" (UID: \"a663edf9-fdf4-4520-bc00-96ed2c535c5c\") "
Oct 30 06:13:48.720010 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.719980    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-home" pod "a663edf9-fdf4-4520-bc00-96ed2c535c5c" (UID: "a663edf9-fdf4-4520-bc00-96ed2c535c5c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:13:48.720204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.720183    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tmp-dir" pod "a663edf9-fdf4-4520-bc00-96ed2c535c5c" (UID: "a663edf9-fdf4-4520-bc00-96ed2c535c5c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:13:48.720474 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.720443    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:13:48.720543 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.720480    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:13:48.721957 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.721889    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-model-cache" pod "a663edf9-fdf4-4520-bc00-96ed2c535c5c" (UID: "a663edf9-fdf4-4520-bc00-96ed2c535c5c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:13:48.725560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.723542    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-dshm" pod "a663edf9-fdf4-4520-bc00-96ed2c535c5c" (UID: "a663edf9-fdf4-4520-bc00-96ed2c535c5c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:13:48.725560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.724458    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a663edf9-fdf4-4520-bc00-96ed2c535c5c-kube-api-access-hkzlw" pod "a663edf9-fdf4-4520-bc00-96ed2c535c5c" (UID: "a663edf9-fdf4-4520-bc00-96ed2c535c5c"). InnerVolumeSpecName "kube-api-access-hkzlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:13:48.726184 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.726154    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tls-certs" pod "a663edf9-fdf4-4520-bc00-96ed2c535c5c" (UID: "a663edf9-fdf4-4520-bc00-96ed2c535c5c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:13:48.821990 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.821949    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkzlw\" (UniqueName: \"kubernetes.io/projected/a663edf9-fdf4-4520-bc00-96ed2c535c5c-kube-api-access-hkzlw\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:13:48.821990 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.821979    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:13:48.821990 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.821991    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a663edf9-fdf4-4520-bc00-96ed2c535c5c-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:13:48.821990 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:48.821999    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a663edf9-fdf4-4520-bc00-96ed2c535c5c-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:13:49.072693 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:49.072663    5066 generic.go:356] "Generic (PLEG): container finished" podID="a663edf9-fdf4-4520-bc00-96ed2c535c5c" containerID="aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1" exitCode=0
Oct 30 06:13:49.072870 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:49.072711    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"
Oct 30 06:13:49.072870 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:49.072746    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6" event={"ID":"a663edf9-fdf4-4520-bc00-96ed2c535c5c","Type":"ContainerDied","Data":"aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1"}
Oct 30 06:13:49.072870 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:49.072787    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6" event={"ID":"a663edf9-fdf4-4520-bc00-96ed2c535c5c","Type":"ContainerDied","Data":"0ac09f9a58a01d61880a64b641388509d0ca72261119dea89fb962da2ef9853a"}
Oct 30 06:13:49.072870 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:49.072807    5066 scope.go:122] "RemoveContainer" containerID="aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1"
Oct 30 06:13:49.084062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:49.084044    5066 scope.go:122] "RemoveContainer" containerID="aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1"
Oct 30 06:13:49.084346 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:13:49.084327    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1\": container with ID starting with aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1 not found: ID does not exist" containerID="aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1"
Oct 30 06:13:49.084414 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:49.084360    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1"} err="failed to get container status \"aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1\": rpc error: code = NotFound desc = could not find container \"aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1\": container with ID starting with aae0bc2d08915503ff6bc83127f6e70f7bf6de75d650951d89ed5c2c7fb58ee1 not found: ID does not exist"
Oct 30 06:13:49.132508 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:49.132477    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"]
Oct 30 06:13:49.146009 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:49.145980    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w6-477f6b/auth-test-model-477f6b-kserve-6b4d855665-8xth6"]
Oct 30 06:13:50.223025 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:50.222987    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:50.403223 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:50.403187    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a663edf9-fdf4-4520-bc00-96ed2c535c5c" path="/var/lib/kubelet/pods/a663edf9-fdf4-4520-bc00-96ed2c535c5c/volumes"
Oct 30 06:13:51.814271 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:51.814183    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:53.430917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:53.430878    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:55.138650 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:55.138576    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:56.704874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:56.704833    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:58.182152 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:58.182120    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:13:59.860533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:13:59.860493    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:01.080684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.080652    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:01.243760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.243727    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:01.424753 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.424688    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"]
Oct 30 06:14:01.425535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.425512    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a663edf9-fdf4-4520-bc00-96ed2c535c5c" containerName="main"
Oct 30 06:14:01.425535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.425532    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a663edf9-fdf4-4520-bc00-96ed2c535c5c" containerName="main"
Oct 30 06:14:01.425728 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.425581    5066 cpu_manager.go:414] "containerMap: removing container" podUID="0fe79012-35f8-4236-93ab-3da8fd9fa91d" containerName="test-curl-9210-4e5884"
Oct 30 06:14:01.425728 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.425604    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0fe79012-35f8-4236-93ab-3da8fd9fa91d" containerName="test-curl-9210-4e5884"
Oct 30 06:14:01.425728 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.425716    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="0fe79012-35f8-4236-93ab-3da8fd9fa91d" containerName="test-curl-9210-4e5884"
Oct 30 06:14:01.425728 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.425725    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a663edf9-fdf4-4520-bc00-96ed2c535c5c" containerName="main"
Oct 30 06:14:01.430813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.430796    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"
Oct 30 06:14:01.446737 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.446708    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"]
Oct 30 06:14:01.460709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.456756    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-db0540-dockercfg-tgnr7\""
Oct 30 06:14:01.460709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.456838    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-db0540\""
Oct 30 06:14:01.508042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.508008    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj\" (UID: \"b360e5eb-1915-47c0-9f8f-7adabbb6f52f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"
Oct 30 06:14:01.508665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.508576    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkvw8\" (UniqueName: \"kubernetes.io/projected/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-kube-api-access-rkvw8\") pod \"payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj\" (UID: \"b360e5eb-1915-47c0-9f8f-7adabbb6f52f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"
Oct 30 06:14:01.594517 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.594481    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"]
Oct 30 06:14:01.598993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.598968    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"
Oct 30 06:14:01.610392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.610360    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj\" (UID: \"b360e5eb-1915-47c0-9f8f-7adabbb6f52f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"
Oct 30 06:14:01.610567 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.610537    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2wpf\" (UniqueName: \"kubernetes.io/projected/16484b77-48d2-4522-a074-1d814552fb11-kube-api-access-g2wpf\") pod \"payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj\" (UID: \"16484b77-48d2-4522-a074-1d814552fb11\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"
Oct 30 06:14:01.610707 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.610689    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkvw8\" (UniqueName: \"kubernetes.io/projected/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-kube-api-access-rkvw8\") pod \"payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj\" (UID: \"b360e5eb-1915-47c0-9f8f-7adabbb6f52f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"
Oct 30 06:14:01.610861 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.610844    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16484b77-48d2-4522-a074-1d814552fb11-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj\" (UID: \"16484b77-48d2-4522-a074-1d814552fb11\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"
Oct 30 06:14:01.611105 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.611087    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj\" (UID: \"b360e5eb-1915-47c0-9f8f-7adabbb6f52f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"
Oct 30 06:14:01.619550 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.619513    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"]
Oct 30 06:14:01.639530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.639496    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkvw8\" (UniqueName: \"kubernetes.io/projected/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-kube-api-access-rkvw8\") pod \"payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj\" (UID: \"b360e5eb-1915-47c0-9f8f-7adabbb6f52f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"
Oct 30 06:14:01.712011 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.711921    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16484b77-48d2-4522-a074-1d814552fb11-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj\" (UID: \"16484b77-48d2-4522-a074-1d814552fb11\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"
Oct 30 06:14:01.712205 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.712033    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2wpf\" (UniqueName: \"kubernetes.io/projected/16484b77-48d2-4522-a074-1d814552fb11-kube-api-access-g2wpf\") pod \"payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj\" (UID: \"16484b77-48d2-4522-a074-1d814552fb11\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"
Oct 30 06:14:01.712814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.712792    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16484b77-48d2-4522-a074-1d814552fb11-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj\" (UID: \"16484b77-48d2-4522-a074-1d814552fb11\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"
Oct 30 06:14:01.740776 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.740737    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2wpf\" (UniqueName: \"kubernetes.io/projected/16484b77-48d2-4522-a074-1d814552fb11-kube-api-access-g2wpf\") pod \"payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj\" (UID: \"16484b77-48d2-4522-a074-1d814552fb11\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"
Oct 30 06:14:01.746582 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.746554    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"
Oct 30 06:14:01.914504 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:01.914467    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"
Oct 30 06:14:02.058281 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:14:02.058249    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb360e5eb_1915_47c0_9f8f_7adabbb6f52f.slice/crio-f649242ac6c29632683523089a9a57c1402dba687311f7a7d071a022b93c935c WatchSource:0}: Error finding container f649242ac6c29632683523089a9a57c1402dba687311f7a7d071a022b93c935c: Status 404 returned error can't find the container with id f649242ac6c29632683523089a9a57c1402dba687311f7a7d071a022b93c935c
Oct 30 06:14:02.058515 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:02.058489    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"]
Oct 30 06:14:02.119069 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:02.118990    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:02.175911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:02.175874    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj" event={"ID":"b360e5eb-1915-47c0-9f8f-7adabbb6f52f","Type":"ContainerStarted","Data":"f649242ac6c29632683523089a9a57c1402dba687311f7a7d071a022b93c935c"}
Oct 30 06:14:02.225625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:02.225578    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"]
Oct 30 06:14:02.226509 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:14:02.226484    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16484b77_48d2_4522_a074_1d814552fb11.slice/crio-e3e58126d95e087728a58ee16cbc7fc4908b23eab5df7631834571fe15b3b706 WatchSource:0}: Error finding container e3e58126d95e087728a58ee16cbc7fc4908b23eab5df7631834571fe15b3b706: Status 404 returned error can't find the container with id e3e58126d95e087728a58ee16cbc7fc4908b23eab5df7631834571fe15b3b706
Oct 30 06:14:02.562775 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:14:02.562680    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:14:03.186252 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:03.186216    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj" event={"ID":"16484b77-48d2-4522-a074-1d814552fb11","Type":"ContainerStarted","Data":"e3e58126d95e087728a58ee16cbc7fc4908b23eab5df7631834571fe15b3b706"}
Oct 30 06:14:03.627674 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:03.625735    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:04.195346 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:04.195310    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj" event={"ID":"b360e5eb-1915-47c0-9f8f-7adabbb6f52f","Type":"ContainerStarted","Data":"2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f"}
Oct 30 06:14:04.195873 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:04.195396    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"
Oct 30 06:14:04.197136 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:04.197103    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj" event={"ID":"16484b77-48d2-4522-a074-1d814552fb11","Type":"ContainerStarted","Data":"d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad"}
Oct 30 06:14:04.197287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:04.197158    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"
Oct 30 06:14:04.243560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:04.243505    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj" podStartSLOduration=1.9614953019999999 podStartE2EDuration="3.243490204s" totalImagesPullingTime="1.281994902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:14:04.239698173 +0000 UTC m=+2547.984983518" watchObservedRunningTime="2026-10-30 06:14:04.243490204 +0000 UTC m=+2547.988775536"
Oct 30 06:14:04.286703 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:04.286653    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj" podStartSLOduration=2.170127802 podStartE2EDuration="3.286638746s" totalImagesPullingTime="1.116510944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:14:04.28443313 +0000 UTC m=+2548.029718463" watchObservedRunningTime="2026-10-30 06:14:04.286638746 +0000 UTC m=+2548.031924077"
Oct 30 06:14:05.333675 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:05.333644    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:06.996305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:06.996262    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:08.668127 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:08.668084    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:10.323758 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:10.323720    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:11.568112 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:11.568074    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:11.865934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:11.865847    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"]
Oct 30 06:14:11.873170 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:11.873139    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:11.891796 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:11.891764    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"]
Oct 30 06:14:11.900801 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:11.900774    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-25d881-serving-cert\""
Oct 30 06:14:11.972196 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:11.972163    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:11.972406 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:11.972266    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q65s\" (UniqueName: \"kubernetes.io/projected/552a0155-ae0c-4811-b0ab-d11480e9cbd4-kube-api-access-4q65s\") pod \"maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:11.972406 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:11.972326    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:11.972406 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:11.972360    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/552a0155-ae0c-4811-b0ab-d11480e9cbd4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:12.008828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:12.008789    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:12.074168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:12.074120    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:12.074168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:12.074176    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/552a0155-ae0c-4811-b0ab-d11480e9cbd4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:12.074452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:12.074348    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:12.074452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:12.074441    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q65s\" (UniqueName: \"kubernetes.io/projected/552a0155-ae0c-4811-b0ab-d11480e9cbd4-kube-api-access-4q65s\") pod \"maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:12.075100 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:12.075078    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/552a0155-ae0c-4811-b0ab-d11480e9cbd4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:12.077676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:12.077650    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:12.077804 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:12.077707    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:12.102828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:12.102792    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q65s\" (UniqueName: \"kubernetes.io/projected/552a0155-ae0c-4811-b0ab-d11480e9cbd4-kube-api-access-4q65s\") pod \"maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:12.165416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:12.165387    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:12.198533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:12.198498    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:12.524144 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:12.524113    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"]
Oct 30 06:14:12.525376 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:14:12.525353    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod552a0155_ae0c_4811_b0ab_d11480e9cbd4.slice/crio-782d3a325d62fa05c0f9e6eca4555dced890d606a3168f13e8fee1d18da53ad6 WatchSource:0}: Error finding container 782d3a325d62fa05c0f9e6eca4555dced890d606a3168f13e8fee1d18da53ad6: Status 404 returned error can't find the container with id 782d3a325d62fa05c0f9e6eca4555dced890d606a3168f13e8fee1d18da53ad6
Oct 30 06:14:13.265290 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:13.265178    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m" event={"ID":"552a0155-ae0c-4811-b0ab-d11480e9cbd4","Type":"ContainerStarted","Data":"04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852"}
Oct 30 06:14:13.265290 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:13.265258    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:13.265290 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:13.265285    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m" event={"ID":"552a0155-ae0c-4811-b0ab-d11480e9cbd4","Type":"ContainerStarted","Data":"782d3a325d62fa05c0f9e6eca4555dced890d606a3168f13e8fee1d18da53ad6"}
Oct 30 06:14:13.318362 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:13.318308    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m" podStartSLOduration=1.925204941 podStartE2EDuration="2.318292203s" totalImagesPullingTime="393.087262ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:14:13.318225643 +0000 UTC m=+2557.063510975" watchObservedRunningTime="2026-10-30 06:14:13.318292203 +0000 UTC m=+2557.063577534"
Oct 30 06:14:13.924992 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:13.924947    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:15.206911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:15.206880    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"
Oct 30 06:14:15.207358 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:15.207289    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"
Oct 30 06:14:15.591016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:15.590913    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:16.948179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:16.948098    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"]
Oct 30 06:14:16.953320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:16.953291    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:16.967698 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:16.967677    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-metrics-certs\") pod \"maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:16.967831 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:16.967728    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:16.967879 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:16.967855    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w762d\" (UniqueName: \"kubernetes.io/projected/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-kube-api-access-w762d\") pod \"maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:16.967914 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:16.967892    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:16.980636 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:16.977626    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"]
Oct 30 06:14:16.985560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:16.985530    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-dddc53-serving-cert\""
Oct 30 06:14:17.069865 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:17.069835    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-metrics-certs\") pod \"maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:17.070049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:17.069879    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:17.070049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:17.069928    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w762d\" (UniqueName: \"kubernetes.io/projected/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-kube-api-access-w762d\") pod \"maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:17.070049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:17.069949    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:17.070555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:17.070530    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:17.072547 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:17.072521    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:17.072673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:17.072571    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-metrics-certs\") pod \"maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:17.104090 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:17.104064    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w762d\" (UniqueName: \"kubernetes.io/projected/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-kube-api-access-w762d\") pod \"maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:17.105887 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:17.105858    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:17.270726 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:17.270616    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:17.271443 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:17.271423    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:17.585194 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:14:17.585163    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc634e3e8_f5df_4c8b_a929_ac74517bd9e8.slice/crio-2c2ecf0107ac60eb370c5e970f9f14a6ef8be820480c8791b94727563137b61e WatchSource:0}: Error finding container 2c2ecf0107ac60eb370c5e970f9f14a6ef8be820480c8791b94727563137b61e: Status 404 returned error can't find the container with id 2c2ecf0107ac60eb370c5e970f9f14a6ef8be820480c8791b94727563137b61e
Oct 30 06:14:17.586280 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:17.586244    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"]
Oct 30 06:14:18.323515 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:18.323413    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4" event={"ID":"c634e3e8-f5df-4c8b-a929-ac74517bd9e8","Type":"ContainerStarted","Data":"d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be"}
Oct 30 06:14:18.323515 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:18.323468    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4" event={"ID":"c634e3e8-f5df-4c8b-a929-ac74517bd9e8","Type":"ContainerStarted","Data":"2c2ecf0107ac60eb370c5e970f9f14a6ef8be820480c8791b94727563137b61e"}
Oct 30 06:14:18.324101 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:18.323518    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:18.370193 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:18.370130    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4" podStartSLOduration=2.037846679 podStartE2EDuration="2.370108533s" totalImagesPullingTime="332.261854ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:14:18.370077069 +0000 UTC m=+2562.115362402" watchObservedRunningTime="2026-10-30 06:14:18.370108533 +0000 UTC m=+2562.115393866"
Oct 30 06:14:19.060279 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:19.060253    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:19.277250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:19.277214    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:14:20.731943 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:20.731898    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:21.076885 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:21.076795    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:21.552789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:21.552754    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:21.799902 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:21.799856    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:22.505469 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:22.505386    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:22.525877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:22.525845    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"]
Oct 30 06:14:22.672439 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:22.672409    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:24.327099 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:24.327066    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:24.345061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:24.345031    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:14:25.001052 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:25.001016    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:25.181582 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:25.181550    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:26.700936 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:26.700900    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:26.869574 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:26.869537    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:28.541654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:28.541583    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:30.056628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:30.052297    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:31.774287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:31.774250    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:32.499561 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:32.499528    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:32.554000 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:14:32.553969    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:14:33.378888 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:33.378848    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:33.539874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:33.539847    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:34.963783 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:34.963740    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:36.584899 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:36.584867    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:37.556842 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:37.556807    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp" podUID="c1744a5e-e05e-4b57-aa48-3bfa02672ee4" containerName="main" containerID="cri-o://531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802" gracePeriod=45
Oct 30 06:14:37.845744 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:37.845714    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:14:38.003467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.003437    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tmp-dir\") pod \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") "
Oct 30 06:14:38.003725 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.003563    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tls-certs\" (UniqueName: \"kubernetes.io/secret/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tls-certs\") pod \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") "
Oct 30 06:14:38.003725 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.003696    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-dshm\") pod \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") "
Oct 30 06:14:38.003725 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.003725    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-kube-api-access-bvdl2\" (UniqueName: \"kubernetes.io/projected/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-kube-api-access-bvdl2\") pod \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") "
Oct 30 06:14:38.003900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.003771    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-home\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-home\") pod \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") "
Oct 30 06:14:38.003900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.003767    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tmp-dir" pod "c1744a5e-e05e-4b57-aa48-3bfa02672ee4" (UID: "c1744a5e-e05e-4b57-aa48-3bfa02672ee4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:14:38.003900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.003872    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-model-cache\") pod \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\" (UID: \"c1744a5e-e05e-4b57-aa48-3bfa02672ee4\") "
Oct 30 06:14:38.004070 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.003963    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-home" pod "c1744a5e-e05e-4b57-aa48-3bfa02672ee4" (UID: "c1744a5e-e05e-4b57-aa48-3bfa02672ee4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:14:38.004277 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.004250    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-model-cache" pod "c1744a5e-e05e-4b57-aa48-3bfa02672ee4" (UID: "c1744a5e-e05e-4b57-aa48-3bfa02672ee4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:14:38.004739 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.004716    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:14:38.004906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.004745    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:14:38.004906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.004761    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:14:38.006242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.006221    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tls-certs" pod "c1744a5e-e05e-4b57-aa48-3bfa02672ee4" (UID: "c1744a5e-e05e-4b57-aa48-3bfa02672ee4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:14:38.006504 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.006489    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-dshm" pod "c1744a5e-e05e-4b57-aa48-3bfa02672ee4" (UID: "c1744a5e-e05e-4b57-aa48-3bfa02672ee4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:14:38.006573 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.006527    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-kube-api-access-bvdl2" pod "c1744a5e-e05e-4b57-aa48-3bfa02672ee4" (UID: "c1744a5e-e05e-4b57-aa48-3bfa02672ee4"). InnerVolumeSpecName "kube-api-access-bvdl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:14:38.106427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.106337    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:14:38.106427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.106374    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvdl2\" (UniqueName: \"kubernetes.io/projected/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-kube-api-access-bvdl2\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:14:38.106427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.106389    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1744a5e-e05e-4b57-aa48-3bfa02672ee4-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:14:38.235187 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.235153    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:38.471307 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.471276    5066 generic.go:356] "Generic (PLEG): container finished" podID="c1744a5e-e05e-4b57-aa48-3bfa02672ee4" containerID="531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802" exitCode=0
Oct 30 06:14:38.471516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.471318    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp" event={"ID":"c1744a5e-e05e-4b57-aa48-3bfa02672ee4","Type":"ContainerDied","Data":"531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802"}
Oct 30 06:14:38.471516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.471335    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"
Oct 30 06:14:38.471516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.471356    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp" event={"ID":"c1744a5e-e05e-4b57-aa48-3bfa02672ee4","Type":"ContainerDied","Data":"f4ee10e8e4651b1d9887f30a34ae1f7c5ab453e3fe25de9450cf2e203d3ab884"}
Oct 30 06:14:38.471516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.471379    5066 scope.go:122] "RemoveContainer" containerID="531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802"
Oct 30 06:14:38.482292 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.482273    5066 scope.go:122] "RemoveContainer" containerID="531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802"
Oct 30 06:14:38.482610 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:14:38.482573    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802\": container with ID starting with 531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802 not found: ID does not exist" containerID="531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802"
Oct 30 06:14:38.482700 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.482640    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802"} err="failed to get container status \"531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802\": rpc error: code = NotFound desc = could not find container \"531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802\": container with ID starting with 531e696039dbb8a0085f542a4ece9f1d50cc82cda4ac037ce72a8f06204eb802 not found: ID does not exist"
Oct 30 06:14:38.531528 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.531495    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"]
Oct 30 06:14:38.545240 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:38.545207    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w4-815776/sub-test-model-815776-kserve-59bff6cd69-7skwp"]
Oct 30 06:14:39.920905 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:39.920863    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:40.404724 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:40.404697    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1744a5e-e05e-4b57-aa48-3bfa02672ee4" path="/var/lib/kubelet/pods/c1744a5e-e05e-4b57-aa48-3bfa02672ee4/volumes"
Oct 30 06:14:41.594623 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:41.594493    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:42.059611 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:42.059078    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:42.363488 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:42.363400    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:42.562094 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:42.562065    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:43.422765 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:43.422733    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:43.613001 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:43.612962    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:44.319212 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:44.319177    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:46.043104 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:46.043065    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:46.954601 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:46.954550    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:47.147886 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:47.147851    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:48.086940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:48.086907    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:48.095874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:48.095839    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"]
Oct 30 06:14:48.257956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:48.257924    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:48.762827 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:48.762794    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:49.353885 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:49.353809    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:49.538388 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:49.538322    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:50.072128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:50.071932    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:50.263209 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:50.263177    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:50.436637 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:50.436600    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:52.059720 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:52.059682    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:54.055334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:54.051790    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:54.246226 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:54.246186    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:55.819216 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:55.819180    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:56.045409 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:56.045361    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:56.657772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:56.657733    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:57.632166 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:57.632057    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:58.125648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:58.125572    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:58.610625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:58.610549    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:14:58.789108 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:14:58.789072    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:00.320109 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.320076    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"]
Oct 30 06:15:00.320968 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.320946    5066 cpu_manager.go:414] "containerMap: removing container" podUID="c1744a5e-e05e-4b57-aa48-3bfa02672ee4" containerName="main"
Oct 30 06:15:00.320968 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.320968    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1744a5e-e05e-4b57-aa48-3bfa02672ee4" containerName="main"
Oct 30 06:15:00.321223 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.321208    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1744a5e-e05e-4b57-aa48-3bfa02672ee4" containerName="main"
Oct 30 06:15:00.326102 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.326067    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"]
Oct 30 06:15:00.326250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.326229    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"
Oct 30 06:15:00.331141 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.331120    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"
Oct 30 06:15:00.353438 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.353416    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"]
Oct 30 06:15:00.359741 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.359718    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"]
Oct 30 06:15:00.360054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.359983    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"
Oct 30 06:15:00.367035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.366969    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"]
Oct 30 06:15:00.369093 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.369067    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"]
Oct 30 06:15:00.375256 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.373409    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:15:00.376705 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.376685    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"
Oct 30 06:15:00.381623 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.381237    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"]
Oct 30 06:15:00.437410 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.437353    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ds4v\" (UniqueName: \"kubernetes.io/projected/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-kube-api-access-6ds4v\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g\" (UID: \"8015f2f2-498e-4eb2-92b5-e5cd89b3352a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"
Oct 30 06:15:00.437887 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.437839    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ljk8\" (UniqueName: \"kubernetes.io/projected/f31f6f0e-e534-48bd-bf2c-9f49feda0561-kube-api-access-4ljk8\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr\" (UID: \"f31f6f0e-e534-48bd-bf2c-9f49feda0561\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"
Oct 30 06:15:00.439316 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.438108    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f31f6f0e-e534-48bd-bf2c-9f49feda0561-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr\" (UID: \"f31f6f0e-e534-48bd-bf2c-9f49feda0561\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"
Oct 30 06:15:00.439950 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.439927    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g\" (UID: \"8015f2f2-498e-4eb2-92b5-e5cd89b3352a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"
Oct 30 06:15:00.452787 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.450480    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:00.477634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.476280    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"]
Oct 30 06:15:00.542278 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.542233    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rhpq\" (UniqueName: \"kubernetes.io/projected/932116d1-8f96-472d-8dd1-5a08f275db18-kube-api-access-6rhpq\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn\" (UID: \"932116d1-8f96-472d-8dd1-5a08f275db18\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"
Oct 30 06:15:00.542468 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.542321    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b6bd70-d8ba-47d4-acb0-580158cb262c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9\" (UID: \"72b6bd70-d8ba-47d4-acb0-580158cb262c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"
Oct 30 06:15:00.542517 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.542496    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g\" (UID: \"8015f2f2-498e-4eb2-92b5-e5cd89b3352a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"
Oct 30 06:15:00.542659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.542565    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ds4v\" (UniqueName: \"kubernetes.io/projected/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-kube-api-access-6ds4v\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g\" (UID: \"8015f2f2-498e-4eb2-92b5-e5cd89b3352a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"
Oct 30 06:15:00.542659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.542653    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wmtg\" (UniqueName: \"kubernetes.io/projected/72b6bd70-d8ba-47d4-acb0-580158cb262c-kube-api-access-6wmtg\") pod \"maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9\" (UID: \"72b6bd70-d8ba-47d4-acb0-580158cb262c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"
Oct 30 06:15:00.542828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.542800    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ljk8\" (UniqueName: \"kubernetes.io/projected/f31f6f0e-e534-48bd-bf2c-9f49feda0561-kube-api-access-4ljk8\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr\" (UID: \"f31f6f0e-e534-48bd-bf2c-9f49feda0561\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"
Oct 30 06:15:00.542903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.542884    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/932116d1-8f96-472d-8dd1-5a08f275db18-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn\" (UID: \"932116d1-8f96-472d-8dd1-5a08f275db18\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"
Oct 30 06:15:00.543352 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.543067    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f31f6f0e-e534-48bd-bf2c-9f49feda0561-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr\" (UID: \"f31f6f0e-e534-48bd-bf2c-9f49feda0561\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"
Oct 30 06:15:00.543352 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.543332    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g\" (UID: \"8015f2f2-498e-4eb2-92b5-e5cd89b3352a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"
Oct 30 06:15:00.543894 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.543872    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f31f6f0e-e534-48bd-bf2c-9f49feda0561-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr\" (UID: \"f31f6f0e-e534-48bd-bf2c-9f49feda0561\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"
Oct 30 06:15:00.573877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.573793    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ds4v\" (UniqueName: \"kubernetes.io/projected/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-kube-api-access-6ds4v\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g\" (UID: \"8015f2f2-498e-4eb2-92b5-e5cd89b3352a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"
Oct 30 06:15:00.576407 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.576384    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ljk8\" (UniqueName: \"kubernetes.io/projected/f31f6f0e-e534-48bd-bf2c-9f49feda0561-kube-api-access-4ljk8\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr\" (UID: \"f31f6f0e-e534-48bd-bf2c-9f49feda0561\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"
Oct 30 06:15:00.645142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.645081    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wmtg\" (UniqueName: \"kubernetes.io/projected/72b6bd70-d8ba-47d4-acb0-580158cb262c-kube-api-access-6wmtg\") pod \"maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9\" (UID: \"72b6bd70-d8ba-47d4-acb0-580158cb262c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"
Oct 30 06:15:00.645346 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.645210    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/932116d1-8f96-472d-8dd1-5a08f275db18-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn\" (UID: \"932116d1-8f96-472d-8dd1-5a08f275db18\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"
Oct 30 06:15:00.645414 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.645358    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rhpq\" (UniqueName: \"kubernetes.io/projected/932116d1-8f96-472d-8dd1-5a08f275db18-kube-api-access-6rhpq\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn\" (UID: \"932116d1-8f96-472d-8dd1-5a08f275db18\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"
Oct 30 06:15:00.645468 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.645427    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b6bd70-d8ba-47d4-acb0-580158cb262c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9\" (UID: \"72b6bd70-d8ba-47d4-acb0-580158cb262c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"
Oct 30 06:15:00.646207 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.646179    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/932116d1-8f96-472d-8dd1-5a08f275db18-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn\" (UID: \"932116d1-8f96-472d-8dd1-5a08f275db18\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"
Oct 30 06:15:00.646347 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.646215    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b6bd70-d8ba-47d4-acb0-580158cb262c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9\" (UID: \"72b6bd70-d8ba-47d4-acb0-580158cb262c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"
Oct 30 06:15:00.649798 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.649773    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"
Oct 30 06:15:00.659813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.659788    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"
Oct 30 06:15:00.686492 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.686457    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wmtg\" (UniqueName: \"kubernetes.io/projected/72b6bd70-d8ba-47d4-acb0-580158cb262c-kube-api-access-6wmtg\") pod \"maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9\" (UID: \"72b6bd70-d8ba-47d4-acb0-580158cb262c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"
Oct 30 06:15:00.686682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.686669    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rhpq\" (UniqueName: \"kubernetes.io/projected/932116d1-8f96-472d-8dd1-5a08f275db18-kube-api-access-6rhpq\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn\" (UID: \"932116d1-8f96-472d-8dd1-5a08f275db18\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"
Oct 30 06:15:00.718744 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.717676    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"
Oct 30 06:15:00.976351 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.976321    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"]
Oct 30 06:15:00.977579 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:15:00.977551    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8015f2f2_498e_4eb2_92b5_e5cd89b3352a.slice/crio-37fa45bbb8cc78533e81052a2dcd424fa74f373bc5cd53dd1213dd6b46cbaa3c WatchSource:0}: Error finding container 37fa45bbb8cc78533e81052a2dcd424fa74f373bc5cd53dd1213dd6b46cbaa3c: Status 404 returned error can't find the container with id 37fa45bbb8cc78533e81052a2dcd424fa74f373bc5cd53dd1213dd6b46cbaa3c
Oct 30 06:15:00.985526 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.985479    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"
Oct 30 06:15:00.989242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:00.989221    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"]
Oct 30 06:15:00.991992 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:15:00.991970    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf31f6f0e_e534_48bd_bf2c_9f49feda0561.slice/crio-841fef2cb27ff951742432abbbc7e34a9a6fdc8ddcefb00f7eecfe1f63232a51 WatchSource:0}: Error finding container 841fef2cb27ff951742432abbbc7e34a9a6fdc8ddcefb00f7eecfe1f63232a51: Status 404 returned error can't find the container with id 841fef2cb27ff951742432abbbc7e34a9a6fdc8ddcefb00f7eecfe1f63232a51
Oct 30 06:15:01.058249 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:01.057740    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"]
Oct 30 06:15:01.060539 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:15:01.060511    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b6bd70_d8ba_47d4_acb0_580158cb262c.slice/crio-da5bf169989b5fdfbf6f83b5846f3de023729cd7d2e41a76cd6f554619cbdcd6 WatchSource:0}: Error finding container da5bf169989b5fdfbf6f83b5846f3de023729cd7d2e41a76cd6f554619cbdcd6: Status 404 returned error can't find the container with id da5bf169989b5fdfbf6f83b5846f3de023729cd7d2e41a76cd6f554619cbdcd6
Oct 30 06:15:01.304069 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:01.304040    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"]
Oct 30 06:15:01.304188 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:15:01.304164    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod932116d1_8f96_472d_8dd1_5a08f275db18.slice/crio-aac5dbf385f2c1d7672cdd551124eaef3cda4efaeebd21bc7c1446caaf1f0e2b WatchSource:0}: Error finding container aac5dbf385f2c1d7672cdd551124eaef3cda4efaeebd21bc7c1446caaf1f0e2b: Status 404 returned error can't find the container with id aac5dbf385f2c1d7672cdd551124eaef3cda4efaeebd21bc7c1446caaf1f0e2b
Oct 30 06:15:01.661363 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:01.661331    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn" event={"ID":"932116d1-8f96-472d-8dd1-5a08f275db18","Type":"ContainerStarted","Data":"aac5dbf385f2c1d7672cdd551124eaef3cda4efaeebd21bc7c1446caaf1f0e2b"}
Oct 30 06:15:01.663081 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:01.663060    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr" event={"ID":"f31f6f0e-e534-48bd-bf2c-9f49feda0561","Type":"ContainerStarted","Data":"16f0996eb504c9f780412d6769b69562094b47e02a4cac6ce95a2342a042aa6b"}
Oct 30 06:15:01.663199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:01.663100    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr" event={"ID":"f31f6f0e-e534-48bd-bf2c-9f49feda0561","Type":"ContainerStarted","Data":"841fef2cb27ff951742432abbbc7e34a9a6fdc8ddcefb00f7eecfe1f63232a51"}
Oct 30 06:15:01.664669 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:01.664640    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g" event={"ID":"8015f2f2-498e-4eb2-92b5-e5cd89b3352a","Type":"ContainerStarted","Data":"71e9b683b1796bc5ca08dfe701ab35c5e14da14647b3881d89a275b56519e403"}
Oct 30 06:15:01.664787 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:01.664692    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g" event={"ID":"8015f2f2-498e-4eb2-92b5-e5cd89b3352a","Type":"ContainerStarted","Data":"37fa45bbb8cc78533e81052a2dcd424fa74f373bc5cd53dd1213dd6b46cbaa3c"}
Oct 30 06:15:01.666536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:01.666515    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9" event={"ID":"72b6bd70-d8ba-47d4-acb0-580158cb262c","Type":"ContainerStarted","Data":"ba3d0e288b2c86bec60fe37106162766531c0b1f117549c1f24ddf7d444eb0cf"}
Oct 30 06:15:01.666676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:01.666549    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9" event={"ID":"72b6bd70-d8ba-47d4-acb0-580158cb262c","Type":"ContainerStarted","Data":"da5bf169989b5fdfbf6f83b5846f3de023729cd7d2e41a76cd6f554619cbdcd6"}
Oct 30 06:15:01.710071 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:01.710017    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr" podStartSLOduration=1.710003969 podStartE2EDuration="1.710003969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:15:01.706322102 +0000 UTC m=+2605.451607434" watchObservedRunningTime="2026-10-30 06:15:01.710003969 +0000 UTC m=+2605.455289301"
Oct 30 06:15:01.754166 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:01.754121    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g" podStartSLOduration=1.754106684 podStartE2EDuration="1.754106684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:15:01.750102147 +0000 UTC m=+2605.495387491" watchObservedRunningTime="2026-10-30 06:15:01.754106684 +0000 UTC m=+2605.499392016"
Oct 30 06:15:01.797919 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:01.797819    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9" podStartSLOduration=1.79780475 podStartE2EDuration="1.79780475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:15:01.793762255 +0000 UTC m=+2605.539047589" watchObservedRunningTime="2026-10-30 06:15:01.79780475 +0000 UTC m=+2605.543090081"
Oct 30 06:15:02.391438 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.391291    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:02.554633 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:02.554502    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:15:02.583876 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.583845    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:02.677000 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.676965    5066 generic.go:356] "Generic (PLEG): container finished" podID="72b6bd70-d8ba-47d4-acb0-580158cb262c" containerID="ba3d0e288b2c86bec60fe37106162766531c0b1f117549c1f24ddf7d444eb0cf" exitCode=60
Oct 30 06:15:02.677453 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.677035    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9" event={"ID":"72b6bd70-d8ba-47d4-acb0-580158cb262c","Type":"ContainerDied","Data":"ba3d0e288b2c86bec60fe37106162766531c0b1f117549c1f24ddf7d444eb0cf"}
Oct 30 06:15:02.677506 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.677494    5066 scope.go:122] "RemoveContainer" containerID="ba3d0e288b2c86bec60fe37106162766531c0b1f117549c1f24ddf7d444eb0cf"
Oct 30 06:15:02.679156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.679122    5066 generic.go:356] "Generic (PLEG): container finished" podID="932116d1-8f96-472d-8dd1-5a08f275db18" containerID="04f050839d537cf30aa0a6324605278425004357139fbc67a144fdffc9080aeb" exitCode=60
Oct 30 06:15:02.679294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.679230    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn" event={"ID":"932116d1-8f96-472d-8dd1-5a08f275db18","Type":"ContainerDied","Data":"04f050839d537cf30aa0a6324605278425004357139fbc67a144fdffc9080aeb"}
Oct 30 06:15:02.679581 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.679443    5066 scope.go:122] "RemoveContainer" containerID="04f050839d537cf30aa0a6324605278425004357139fbc67a144fdffc9080aeb"
Oct 30 06:15:02.683647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.683509    5066 generic.go:356] "Generic (PLEG): container finished" podID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" containerID="16f0996eb504c9f780412d6769b69562094b47e02a4cac6ce95a2342a042aa6b" exitCode=60
Oct 30 06:15:02.683647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.683573    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr" event={"ID":"f31f6f0e-e534-48bd-bf2c-9f49feda0561","Type":"ContainerDied","Data":"16f0996eb504c9f780412d6769b69562094b47e02a4cac6ce95a2342a042aa6b"}
Oct 30 06:15:02.683889 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.683823    5066 scope.go:122] "RemoveContainer" containerID="16f0996eb504c9f780412d6769b69562094b47e02a4cac6ce95a2342a042aa6b"
Oct 30 06:15:02.698310 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.698278    5066 generic.go:356] "Generic (PLEG): container finished" podID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" containerID="71e9b683b1796bc5ca08dfe701ab35c5e14da14647b3881d89a275b56519e403" exitCode=60
Oct 30 06:15:02.698462 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.698432    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g" event={"ID":"8015f2f2-498e-4eb2-92b5-e5cd89b3352a","Type":"ContainerDied","Data":"71e9b683b1796bc5ca08dfe701ab35c5e14da14647b3881d89a275b56519e403"}
Oct 30 06:15:02.698892 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.698870    5066 scope.go:122] "RemoveContainer" containerID="71e9b683b1796bc5ca08dfe701ab35c5e14da14647b3881d89a275b56519e403"
Oct 30 06:15:02.787464 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:02.787436    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:03.215056 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.214976    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b" podUID="7c9c9518-ad3f-478c-964e-2987da51aaa7" containerName="main" containerID="cri-o://2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868" gracePeriod=45
Oct 30 06:15:03.275909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.275867    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b" podUID="7c9c9518-ad3f-478c-964e-2987da51aaa7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.152:8000/ready\": dial tcp 10.132.0.152:8000: connect: connection refused"
Oct 30 06:15:03.526396 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.526368    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:15:03.594581 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.594548    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c9c9518-ad3f-478c-964e-2987da51aaa7-tls-certs\" (UniqueName: \"kubernetes.io/secret/7c9c9518-ad3f-478c-964e-2987da51aaa7-tls-certs\") pod \"7c9c9518-ad3f-478c-964e-2987da51aaa7\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") "
Oct 30 06:15:03.594805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.594664    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-tmp-dir\") pod \"7c9c9518-ad3f-478c-964e-2987da51aaa7\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") "
Oct 30 06:15:03.594805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.594738    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-model-cache\") pod \"7c9c9518-ad3f-478c-964e-2987da51aaa7\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") "
Oct 30 06:15:03.594805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.594786    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c9c9518-ad3f-478c-964e-2987da51aaa7-kube-api-access-27szc\" (UniqueName: \"kubernetes.io/projected/7c9c9518-ad3f-478c-964e-2987da51aaa7-kube-api-access-27szc\") pod \"7c9c9518-ad3f-478c-964e-2987da51aaa7\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") "
Oct 30 06:15:03.594980 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.594854    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-home\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-home\") pod \"7c9c9518-ad3f-478c-964e-2987da51aaa7\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") "
Oct 30 06:15:03.594980 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.594963    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-dshm\") pod \"7c9c9518-ad3f-478c-964e-2987da51aaa7\" (UID: \"7c9c9518-ad3f-478c-964e-2987da51aaa7\") "
Oct 30 06:15:03.596021 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.595937    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-model-cache" pod "7c9c9518-ad3f-478c-964e-2987da51aaa7" (UID: "7c9c9518-ad3f-478c-964e-2987da51aaa7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:15:03.596156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.596051    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-tmp-dir" pod "7c9c9518-ad3f-478c-964e-2987da51aaa7" (UID: "7c9c9518-ad3f-478c-964e-2987da51aaa7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:15:03.596305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.596263    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-home" pod "7c9c9518-ad3f-478c-964e-2987da51aaa7" (UID: "7c9c9518-ad3f-478c-964e-2987da51aaa7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:15:03.598137 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.598112    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-dshm" pod "7c9c9518-ad3f-478c-964e-2987da51aaa7" (UID: "7c9c9518-ad3f-478c-964e-2987da51aaa7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:15:03.598567 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.598545    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9c9518-ad3f-478c-964e-2987da51aaa7-tls-certs" pod "7c9c9518-ad3f-478c-964e-2987da51aaa7" (UID: "7c9c9518-ad3f-478c-964e-2987da51aaa7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:15:03.598713 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.598693    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c9c9518-ad3f-478c-964e-2987da51aaa7-kube-api-access-27szc" pod "7c9c9518-ad3f-478c-964e-2987da51aaa7" (UID: "7c9c9518-ad3f-478c-964e-2987da51aaa7"). InnerVolumeSpecName "kube-api-access-27szc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:15:03.696384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.696343    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27szc\" (UniqueName: \"kubernetes.io/projected/7c9c9518-ad3f-478c-964e-2987da51aaa7-kube-api-access-27szc\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:03.696384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.696376    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:03.696384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.696392    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:03.696921 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.696403    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c9c9518-ad3f-478c-964e-2987da51aaa7-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:03.696921 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.696416    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:03.696921 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.696426    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c9c9518-ad3f-478c-964e-2987da51aaa7-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:03.707400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.707365    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr" event={"ID":"f31f6f0e-e534-48bd-bf2c-9f49feda0561","Type":"ContainerStarted","Data":"db54f494f09de7f5b4a9b346908fe3ad9921a9010c38c99efa6c89838524d213"}
Oct 30 06:15:03.709873 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.709844    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g" event={"ID":"8015f2f2-498e-4eb2-92b5-e5cd89b3352a","Type":"ContainerStarted","Data":"6db7ddfb66609fb944255101c8d60e3385b1f95f711fe5899895a6183a4c7a09"}
Oct 30 06:15:03.711681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.711653    5066 generic.go:356] "Generic (PLEG): container finished" podID="7c9c9518-ad3f-478c-964e-2987da51aaa7" containerID="2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868" exitCode=0
Oct 30 06:15:03.711837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.711717    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"
Oct 30 06:15:03.711837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.711741    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b" event={"ID":"7c9c9518-ad3f-478c-964e-2987da51aaa7","Type":"ContainerDied","Data":"2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868"}
Oct 30 06:15:03.711837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.711794    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b" event={"ID":"7c9c9518-ad3f-478c-964e-2987da51aaa7","Type":"ContainerDied","Data":"bb7a0428214c65b2721e66615c9f64581438397d9f4caed08b8506e117cc24d9"}
Oct 30 06:15:03.711837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.711815    5066 scope.go:122] "RemoveContainer" containerID="2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868"
Oct 30 06:15:03.715431 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.715407    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9" event={"ID":"72b6bd70-d8ba-47d4-acb0-580158cb262c","Type":"ContainerStarted","Data":"d4dc10c3014094ba26416dd633efa8a5149755fe2b09d2c2932ca4011510c1a3"}
Oct 30 06:15:03.718837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.718815    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn" event={"ID":"932116d1-8f96-472d-8dd1-5a08f275db18","Type":"ContainerStarted","Data":"92d3a805ba76416006fff9fa3aa5eadf7bf984e47afd3fe82f0a72e22941f4ee"}
Oct 30 06:15:03.734978 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.734781    5066 scope.go:122] "RemoveContainer" containerID="2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868"
Oct 30 06:15:03.735161 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:03.735137    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868\": container with ID starting with 2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868 not found: ID does not exist" containerID="2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868"
Oct 30 06:15:03.735232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.735167    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868"} err="failed to get container status \"2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868\": rpc error: code = NotFound desc = could not find container \"2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868\": container with ID starting with 2936bcc56958001a4a42c96ba7fc8a0ecb0e422cc716febbb7de71ca88502868 not found: ID does not exist"
Oct 30 06:15:03.884727 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.884670    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn" podStartSLOduration=3.884655516 podStartE2EDuration="3.884655516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:15:03.883410717 +0000 UTC m=+2607.628696084" watchObservedRunningTime="2026-10-30 06:15:03.884655516 +0000 UTC m=+2607.629940847"
Oct 30 06:15:03.969720 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.969686    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"]
Oct 30 06:15:03.989719 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:03.989693    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c73fcb/auth-test-model-c73fcb-kserve-c98776c57-twr2b"]
Oct 30 06:15:04.304198 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.304169    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:04.404769 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.404741    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c9c9518-ad3f-478c-964e-2987da51aaa7" path="/var/lib/kubelet/pods/7c9c9518-ad3f-478c-964e-2987da51aaa7/volumes"
Oct 30 06:15:04.500778 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.500741    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:04.731772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.731738    5066 generic.go:356] "Generic (PLEG): container finished" podID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" containerID="db54f494f09de7f5b4a9b346908fe3ad9921a9010c38c99efa6c89838524d213" exitCode=60
Oct 30 06:15:04.732304 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.731856    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr" event={"ID":"f31f6f0e-e534-48bd-bf2c-9f49feda0561","Type":"ContainerDied","Data":"db54f494f09de7f5b4a9b346908fe3ad9921a9010c38c99efa6c89838524d213"}
Oct 30 06:15:04.732304 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.731916    5066 scope.go:122] "RemoveContainer" containerID="16f0996eb504c9f780412d6769b69562094b47e02a4cac6ce95a2342a042aa6b"
Oct 30 06:15:04.732304 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.732239    5066 scope.go:122] "RemoveContainer" containerID="db54f494f09de7f5b4a9b346908fe3ad9921a9010c38c99efa6c89838524d213"
Oct 30 06:15:04.732842 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:04.732617    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr_odh-ai-gateway-infra(f31f6f0e-e534-48bd-bf2c-9f49feda0561)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561"
Oct 30 06:15:04.745416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.745390    5066 generic.go:356] "Generic (PLEG): container finished" podID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" containerID="6db7ddfb66609fb944255101c8d60e3385b1f95f711fe5899895a6183a4c7a09" exitCode=60
Oct 30 06:15:04.745566 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.745465    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g" event={"ID":"8015f2f2-498e-4eb2-92b5-e5cd89b3352a","Type":"ContainerDied","Data":"6db7ddfb66609fb944255101c8d60e3385b1f95f711fe5899895a6183a4c7a09"}
Oct 30 06:15:04.745930 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.745895    5066 scope.go:122] "RemoveContainer" containerID="6db7ddfb66609fb944255101c8d60e3385b1f95f711fe5899895a6183a4c7a09"
Oct 30 06:15:04.746266 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:04.746242    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g_odh-ai-gateway-infra(8015f2f2-498e-4eb2-92b5-e5cd89b3352a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a"
Oct 30 06:15:04.758398 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.757479    5066 scope.go:122] "RemoveContainer" containerID="71e9b683b1796bc5ca08dfe701ab35c5e14da14647b3881d89a275b56519e403"
Oct 30 06:15:04.764708 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.764584    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9" event={"ID":"72b6bd70-d8ba-47d4-acb0-580158cb262c","Type":"ContainerDied","Data":"d4dc10c3014094ba26416dd633efa8a5149755fe2b09d2c2932ca4011510c1a3"}
Oct 30 06:15:04.764708 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.764453    5066 generic.go:356] "Generic (PLEG): container finished" podID="72b6bd70-d8ba-47d4-acb0-580158cb262c" containerID="d4dc10c3014094ba26416dd633efa8a5149755fe2b09d2c2932ca4011510c1a3" exitCode=60
Oct 30 06:15:04.765348 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.765327    5066 scope.go:122] "RemoveContainer" containerID="d4dc10c3014094ba26416dd633efa8a5149755fe2b09d2c2932ca4011510c1a3"
Oct 30 06:15:04.770288 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:04.765814    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9_odh-ai-gateway-infra(72b6bd70-d8ba-47d4-acb0-580158cb262c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c"
Oct 30 06:15:04.770288 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.768657    5066 generic.go:356] "Generic (PLEG): container finished" podID="932116d1-8f96-472d-8dd1-5a08f275db18" containerID="92d3a805ba76416006fff9fa3aa5eadf7bf984e47afd3fe82f0a72e22941f4ee" exitCode=60
Oct 30 06:15:04.770288 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.768734    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn" event={"ID":"932116d1-8f96-472d-8dd1-5a08f275db18","Type":"ContainerDied","Data":"92d3a805ba76416006fff9fa3aa5eadf7bf984e47afd3fe82f0a72e22941f4ee"}
Oct 30 06:15:04.770288 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.769149    5066 scope.go:122] "RemoveContainer" containerID="92d3a805ba76416006fff9fa3aa5eadf7bf984e47afd3fe82f0a72e22941f4ee"
Oct 30 06:15:04.770288 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:04.769546    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn_odh-ai-gateway-infra(932116d1-8f96-472d-8dd1-5a08f275db18)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn" podUID="932116d1-8f96-472d-8dd1-5a08f275db18"
Oct 30 06:15:04.788271 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.788089    5066 scope.go:122] "RemoveContainer" containerID="ba3d0e288b2c86bec60fe37106162766531c0b1f117549c1f24ddf7d444eb0cf"
Oct 30 06:15:04.812728 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:04.812708    5066 scope.go:122] "RemoveContainer" containerID="04f050839d537cf30aa0a6324605278425004357139fbc67a144fdffc9080aeb"
Oct 30 06:15:05.778248 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:05.778217    5066 scope.go:122] "RemoveContainer" containerID="d4dc10c3014094ba26416dd633efa8a5149755fe2b09d2c2932ca4011510c1a3"
Oct 30 06:15:05.778773 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:05.778635    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9_odh-ai-gateway-infra(72b6bd70-d8ba-47d4-acb0-580158cb262c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c"
Oct 30 06:15:05.779895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:05.779874    5066 scope.go:122] "RemoveContainer" containerID="92d3a805ba76416006fff9fa3aa5eadf7bf984e47afd3fe82f0a72e22941f4ee"
Oct 30 06:15:05.780202 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:05.780180    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn_odh-ai-gateway-infra(932116d1-8f96-472d-8dd1-5a08f275db18)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn" podUID="932116d1-8f96-472d-8dd1-5a08f275db18"
Oct 30 06:15:05.781535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:05.781513    5066 scope.go:122] "RemoveContainer" containerID="db54f494f09de7f5b4a9b346908fe3ad9921a9010c38c99efa6c89838524d213"
Oct 30 06:15:05.781844 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:05.781805    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr_odh-ai-gateway-infra(f31f6f0e-e534-48bd-bf2c-9f49feda0561)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561"
Oct 30 06:15:05.783237 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:05.783211    5066 scope.go:122] "RemoveContainer" containerID="6db7ddfb66609fb944255101c8d60e3385b1f95f711fe5899895a6183a4c7a09"
Oct 30 06:15:05.783468 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:05.783451    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g_odh-ai-gateway-infra(8015f2f2-498e-4eb2-92b5-e5cd89b3352a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a"
Oct 30 06:15:06.386727 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:06.386695    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:07.636072 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:07.636030    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:07.799148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:07.798898    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:07.957866 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:07.957835    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:08.046889 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.046857    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"]
Oct 30 06:15:08.047546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.047532    5066 cpu_manager.go:414] "containerMap: removing container" podUID="7c9c9518-ad3f-478c-964e-2987da51aaa7" containerName="main"
Oct 30 06:15:08.047607 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.047547    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c9c9518-ad3f-478c-964e-2987da51aaa7" containerName="main"
Oct 30 06:15:08.047705 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.047694    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c9c9518-ad3f-478c-964e-2987da51aaa7" containerName="main"
Oct 30 06:15:08.053053 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.053029    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.099614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.095918    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-2d4576-serving-cert\""
Oct 30 06:15:08.171625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.157524    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.171625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.157610    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf5b5a4-32f5-4b5d-b524-632460da3bc1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.171625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.157651    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl56d\" (UniqueName: \"kubernetes.io/projected/abf5b5a4-32f5-4b5d-b524-632460da3bc1-kube-api-access-rl56d\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.171625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.157703    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-metrics-certs\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.204634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.204530    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:08.225484 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.225385    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"]
Oct 30 06:15:08.272660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.265408    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.272660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.265476    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf5b5a4-32f5-4b5d-b524-632460da3bc1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.272660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.265510    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl56d\" (UniqueName: \"kubernetes.io/projected/abf5b5a4-32f5-4b5d-b524-632460da3bc1-kube-api-access-rl56d\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.272660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.265578    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-metrics-certs\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.272660 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:08.272041    5066 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-serving-cert: secret "maas-api-e2e-ipp-b-2d4576-serving-cert" not found
Oct 30 06:15:08.272660 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:08.272132    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-maas-api-tls podName:abf5b5a4-32f5-4b5d-b524-632460da3bc1 nodeName:}" failed. No retries permitted until 2026-10-30 06:15:08.772110158 +0000 UTC m=+2612.517395482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-maas-api-tls") pod "maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk" (UID: "abf5b5a4-32f5-4b5d-b524-632460da3bc1") : secret "maas-api-e2e-ipp-b-2d4576-serving-cert" not found
Oct 30 06:15:08.273487 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.273156    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf5b5a4-32f5-4b5d-b524-632460da3bc1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.276633 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.276405    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-metrics-certs\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.301271 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.301239    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl56d\" (UniqueName: \"kubernetes.io/projected/abf5b5a4-32f5-4b5d-b524-632460da3bc1-kube-api-access-rl56d\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.383935 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.383896    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:08.551290 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.551210    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:08.772795 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.772762    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.775510 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.775481    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:08.967981 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:08.967938    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:09.346612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:09.342801    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"]
Oct 30 06:15:09.818003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:09.817969    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk" event={"ID":"abf5b5a4-32f5-4b5d-b524-632460da3bc1","Type":"ContainerStarted","Data":"b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc"}
Oct 30 06:15:09.818003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:09.818019    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk" event={"ID":"abf5b5a4-32f5-4b5d-b524-632460da3bc1","Type":"ContainerStarted","Data":"ecfb014591568de22c31c47769c6b11d5fdc3176e8fef960db9ea811c6fdab65"}
Oct 30 06:15:09.818573 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:09.818210    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:09.879295 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:09.879234    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk" podStartSLOduration=1.502570768 podStartE2EDuration="1.879217717s" totalImagesPullingTime="376.646949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:15:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:15:09.877055806 +0000 UTC m=+2613.622341143" watchObservedRunningTime="2026-10-30 06:15:09.879217717 +0000 UTC m=+2613.624503048"
Oct 30 06:15:09.944906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:09.944868    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:11.896614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:11.893291    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:13.424653 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:13.423390    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:14.395142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.395103    5066 scope.go:122] "RemoveContainer" containerID="6db7ddfb66609fb944255101c8d60e3385b1f95f711fe5899895a6183a4c7a09"
Oct 30 06:15:14.395426 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.395167    5066 scope.go:122] "RemoveContainer" containerID="d4dc10c3014094ba26416dd633efa8a5149755fe2b09d2c2932ca4011510c1a3"
Oct 30 06:15:14.446278 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.446234    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"]
Oct 30 06:15:14.452414 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.452389    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.473894 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.473862    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-6b5b1e-openshift-default-dockercfg-n2kmp\""
Oct 30 06:15:14.495916 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.495878    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"]
Oct 30 06:15:14.565651 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.565569    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-token\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.565855 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.565821    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-certs\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.565934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.565851    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zn9s\" (UniqueName: \"kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-kube-api-access-9zn9s\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.565934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.565900    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-data\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.566054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.565934    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-podinfo\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.566102 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.566060    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-credential-socket\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.566102 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.566082    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-socket\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.566345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.566325    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istiod-ca-cert\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.566407 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.566356    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-envoy\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.668009 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.667972    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-podinfo\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.668202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.668025    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-credential-socket\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.668202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.668051    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-socket\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.668202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.668125    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istiod-ca-cert\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.668202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.668155    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-envoy\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.668447 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.668286    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-token\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.668447 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.668431    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-certs\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.668561 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.668459    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zn9s\" (UniqueName: \"kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-kube-api-access-9zn9s\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.668712 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.668567    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-credential-socket\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.668712 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.668688    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-socket\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.668926 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.668850    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-certs\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.668926 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.668906    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-data\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.669172 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.669016    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istiod-ca-cert\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.669172 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.669020    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-data\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.670838 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.670800    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-envoy\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.670952 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.670907    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-podinfo\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.697027 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.696999    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-token\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.697553 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.697528    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zn9s\" (UniqueName: \"kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-kube-api-access-9zn9s\") pod \"e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") " pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.773278 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.773242    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:14.861471 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:14.859241    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g" event={"ID":"8015f2f2-498e-4eb2-92b5-e5cd89b3352a","Type":"ContainerStarted","Data":"5f17a1736a61090aa33783f5379c86a15aed4c98e29c57dd219b65937da9ea01"}
Oct 30 06:15:15.109020 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.108150    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:15.117622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.115702    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"]
Oct 30 06:15:15.124646 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:15:15.124499    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53b1f7d3_1fde_456e_92f1_8f45c89036ae.slice/crio-4f57f46ed627110bec0ecf6f1509970c9ef47a5e1ba3004bad90a628b4cc3e0f WatchSource:0}: Error finding container 4f57f46ed627110bec0ecf6f1509970c9ef47a5e1ba3004bad90a628b4cc3e0f: Status 404 returned error can't find the container with id 4f57f46ed627110bec0ecf6f1509970c9ef47a5e1ba3004bad90a628b4cc3e0f
Oct 30 06:15:15.128321 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.128285    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:15:15.128436 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.128377    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:15:15.128436 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.128427    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:15:15.852627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.852533    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:15:15.869570 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.869539    5066 generic.go:356] "Generic (PLEG): container finished" podID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" containerID="5f17a1736a61090aa33783f5379c86a15aed4c98e29c57dd219b65937da9ea01" exitCode=60
Oct 30 06:15:15.869783 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.869623    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g" event={"ID":"8015f2f2-498e-4eb2-92b5-e5cd89b3352a","Type":"ContainerDied","Data":"5f17a1736a61090aa33783f5379c86a15aed4c98e29c57dd219b65937da9ea01"}
Oct 30 06:15:15.869783 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.869687    5066 scope.go:122] "RemoveContainer" containerID="6db7ddfb66609fb944255101c8d60e3385b1f95f711fe5899895a6183a4c7a09"
Oct 30 06:15:15.870206 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.870184    5066 scope.go:122] "RemoveContainer" containerID="5f17a1736a61090aa33783f5379c86a15aed4c98e29c57dd219b65937da9ea01"
Oct 30 06:15:15.870690 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:15.870657    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g_odh-ai-gateway-infra(8015f2f2-498e-4eb2-92b5-e5cd89b3352a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a"
Oct 30 06:15:15.871665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.871641    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" event={"ID":"53b1f7d3-1fde-456e-92f1-8f45c89036ae","Type":"ContainerStarted","Data":"121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd"}
Oct 30 06:15:15.871802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.871691    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" event={"ID":"53b1f7d3-1fde-456e-92f1-8f45c89036ae","Type":"ContainerStarted","Data":"4f57f46ed627110bec0ecf6f1509970c9ef47a5e1ba3004bad90a628b4cc3e0f"}
Oct 30 06:15:15.873794 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.873775    5066 generic.go:356] "Generic (PLEG): container finished" podID="72b6bd70-d8ba-47d4-acb0-580158cb262c" containerID="59b1bc060c805bfb1112cc3449c57109d512f496510d7015aa08d88da4ec9b55" exitCode=60
Oct 30 06:15:15.873898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.873846    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9" event={"ID":"72b6bd70-d8ba-47d4-acb0-580158cb262c","Type":"ContainerDied","Data":"59b1bc060c805bfb1112cc3449c57109d512f496510d7015aa08d88da4ec9b55"}
Oct 30 06:15:15.874239 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.874222    5066 scope.go:122] "RemoveContainer" containerID="59b1bc060c805bfb1112cc3449c57109d512f496510d7015aa08d88da4ec9b55"
Oct 30 06:15:15.874566 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:15.874547    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9_odh-ai-gateway-infra(72b6bd70-d8ba-47d4-acb0-580158cb262c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c"
Oct 30 06:15:15.886760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.886736    5066 scope.go:122] "RemoveContainer" containerID="d4dc10c3014094ba26416dd633efa8a5149755fe2b09d2c2932ca4011510c1a3"
Oct 30 06:15:15.960228 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:15.960188    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"]
Oct 30 06:15:16.065239 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.065185    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" podStartSLOduration=2.065172567 podStartE2EDuration="2.065172567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:15:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:15:16.064425698 +0000 UTC m=+2619.809711031" watchObservedRunningTime="2026-10-30 06:15:16.065172567 +0000 UTC m=+2619.810457898"
Oct 30 06:15:16.398919 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.398890    5066 scope.go:122] "RemoveContainer" containerID="db54f494f09de7f5b4a9b346908fe3ad9921a9010c38c99efa6c89838524d213"
Oct 30 06:15:16.773765 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.773730    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:16.775881 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.775845    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 06:15:16.776058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.775907    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" podUID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 06:15:16.788427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.788398    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"]
Oct 30 06:15:16.794416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.794392    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:16.817391 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.817362    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w5-d1bfce-openshift-default-dockercfg-xxfmp\""
Oct 30 06:15:16.846620 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.844397    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"]
Oct 30 06:15:16.904348 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.904318    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-data\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:16.904742 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.904353    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-token\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:16.904742 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.904469    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-certs\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:16.904742 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.904505    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-podinfo\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:16.904742 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.904617    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l54ss\" (UniqueName: \"kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-kube-api-access-l54ss\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:16.904742 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.904711    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istiod-ca-cert\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:16.904913 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.904805    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-envoy\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:16.904913 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.904828    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-credential-socket\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:16.904913 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:16.904849    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-socket\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.002712 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.002656    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"]
Oct 30 06:15:17.008676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.006516    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-envoy\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.008676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.006571    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-credential-socket\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.008676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.006618    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-socket\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.008676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.006692    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-data\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.008676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.006719    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-token\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.008676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.006802    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-certs\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.008676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.006830    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-podinfo\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.008676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.006958    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l54ss\" (UniqueName: \"kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-kube-api-access-l54ss\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.008676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.007078    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istiod-ca-cert\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.008676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.007968    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istiod-ca-cert\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.008676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.008487    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-certs\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.011334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.010336    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-socket\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.011334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.010498    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-credential-socket\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.011334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.010706    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-data\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.014615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.014127    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-podinfo\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.019634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.017995    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-envoy\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.068120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.068069    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-token\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.069198 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.069110    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l54ss\" (UniqueName: \"kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-kube-api-access-l54ss\") pod \"e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") " pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.112899 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.112849    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:17.217114 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.193799    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"
Oct 30 06:15:17.261661 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.260157    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:17.312051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.312023    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-kube-api-access-6ds4v\" (UniqueName: \"kubernetes.io/projected/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-kube-api-access-6ds4v\") pod \"8015f2f2-498e-4eb2-92b5-e5cd89b3352a\" (UID: \"8015f2f2-498e-4eb2-92b5-e5cd89b3352a\") "
Oct 30 06:15:17.312479 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.312459    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-maas-api-service-ca\") pod \"8015f2f2-498e-4eb2-92b5-e5cd89b3352a\" (UID: \"8015f2f2-498e-4eb2-92b5-e5cd89b3352a\") "
Oct 30 06:15:17.313008 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.312983    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-maas-api-service-ca" pod "8015f2f2-498e-4eb2-92b5-e5cd89b3352a" (UID: "8015f2f2-498e-4eb2-92b5-e5cd89b3352a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:15:17.313215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.313201    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-maas-api-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:17.317085 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.316223    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-kube-api-access-6ds4v" pod "8015f2f2-498e-4eb2-92b5-e5cd89b3352a" (UID: "8015f2f2-498e-4eb2-92b5-e5cd89b3352a"). InnerVolumeSpecName "kube-api-access-6ds4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:15:17.369363 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.369336    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"
Oct 30 06:15:17.398638 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.398559    5066 scope.go:122] "RemoveContainer" containerID="92d3a805ba76416006fff9fa3aa5eadf7bf984e47afd3fe82f0a72e22941f4ee"
Oct 30 06:15:17.415129 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.415092    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ds4v\" (UniqueName: \"kubernetes.io/projected/8015f2f2-498e-4eb2-92b5-e5cd89b3352a-kube-api-access-6ds4v\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:17.517683 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.517647    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72b6bd70-d8ba-47d4-acb0-580158cb262c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b6bd70-d8ba-47d4-acb0-580158cb262c-maas-api-service-ca\") pod \"72b6bd70-d8ba-47d4-acb0-580158cb262c\" (UID: \"72b6bd70-d8ba-47d4-acb0-580158cb262c\") "
Oct 30 06:15:17.517906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.517759    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72b6bd70-d8ba-47d4-acb0-580158cb262c-kube-api-access-6wmtg\" (UniqueName: \"kubernetes.io/projected/72b6bd70-d8ba-47d4-acb0-580158cb262c-kube-api-access-6wmtg\") pod \"72b6bd70-d8ba-47d4-acb0-580158cb262c\" (UID: \"72b6bd70-d8ba-47d4-acb0-580158cb262c\") "
Oct 30 06:15:17.518787 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.518754    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72b6bd70-d8ba-47d4-acb0-580158cb262c-maas-api-service-ca" pod "72b6bd70-d8ba-47d4-acb0-580158cb262c" (UID: "72b6bd70-d8ba-47d4-acb0-580158cb262c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:15:17.552944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.543216    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72b6bd70-d8ba-47d4-acb0-580158cb262c-kube-api-access-6wmtg" pod "72b6bd70-d8ba-47d4-acb0-580158cb262c" (UID: "72b6bd70-d8ba-47d4-acb0-580158cb262c"). InnerVolumeSpecName "kube-api-access-6wmtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:15:17.568334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.568305    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"]
Oct 30 06:15:17.611628 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:15:17.611572    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea38610c_5a57_41c5_bd84_f2c66a5903f6.slice/crio-6904f505399dac4435d7e34f6209326142e9b7c54fe0bcfa1f2f9a8e7e279ca9 WatchSource:0}: Error finding container 6904f505399dac4435d7e34f6209326142e9b7c54fe0bcfa1f2f9a8e7e279ca9: Status 404 returned error can't find the container with id 6904f505399dac4435d7e34f6209326142e9b7c54fe0bcfa1f2f9a8e7e279ca9
Oct 30 06:15:17.614641 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.614608    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:15:17.614719 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.614679    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:15:17.614719 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.614708    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:15:17.620831 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.620205    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72b6bd70-d8ba-47d4-acb0-580158cb262c-maas-api-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:17.620831 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.620226    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wmtg\" (UniqueName: \"kubernetes.io/projected/72b6bd70-d8ba-47d4-acb0-580158cb262c-kube-api-access-6wmtg\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:17.774384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.774351    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 06:15:17.774528 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.774411    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" podUID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 06:15:17.900848 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.900807    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9" event={"ID":"72b6bd70-d8ba-47d4-acb0-580158cb262c","Type":"ContainerDied","Data":"da5bf169989b5fdfbf6f83b5846f3de023729cd7d2e41a76cd6f554619cbdcd6"}
Oct 30 06:15:17.901036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.900888    5066 scope.go:122] "RemoveContainer" containerID="59b1bc060c805bfb1112cc3449c57109d512f496510d7015aa08d88da4ec9b55"
Oct 30 06:15:17.901036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.901026    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"
Oct 30 06:15:17.904712 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.904686    5066 generic.go:356] "Generic (PLEG): container finished" podID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" containerID="c6ea4e7c19ef11db76eed2bd2d11274603e34737e753bf9092ec96d4fbf9fbf3" exitCode=60
Oct 30 06:15:17.905009 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.904741    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr" event={"ID":"f31f6f0e-e534-48bd-bf2c-9f49feda0561","Type":"ContainerDied","Data":"c6ea4e7c19ef11db76eed2bd2d11274603e34737e753bf9092ec96d4fbf9fbf3"}
Oct 30 06:15:17.905146 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.905130    5066 scope.go:122] "RemoveContainer" containerID="c6ea4e7c19ef11db76eed2bd2d11274603e34737e753bf9092ec96d4fbf9fbf3"
Oct 30 06:15:17.905452 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:17.905436    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr_odh-ai-gateway-infra(f31f6f0e-e534-48bd-bf2c-9f49feda0561)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561"
Oct 30 06:15:17.910149 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.910129    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"
Oct 30 06:15:17.910245 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.910176    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g" event={"ID":"8015f2f2-498e-4eb2-92b5-e5cd89b3352a","Type":"ContainerDied","Data":"37fa45bbb8cc78533e81052a2dcd424fa74f373bc5cd53dd1213dd6b46cbaa3c"}
Oct 30 06:15:17.913915 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.913013    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" event={"ID":"ea38610c-5a57-41c5-bd84-f2c66a5903f6","Type":"ContainerStarted","Data":"b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60"}
Oct 30 06:15:17.913915 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.913051    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" event={"ID":"ea38610c-5a57-41c5-bd84-f2c66a5903f6","Type":"ContainerStarted","Data":"6904f505399dac4435d7e34f6209326142e9b7c54fe0bcfa1f2f9a8e7e279ca9"}
Oct 30 06:15:17.917861 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.917214    5066 scope.go:122] "RemoveContainer" containerID="db54f494f09de7f5b4a9b346908fe3ad9921a9010c38c99efa6c89838524d213"
Oct 30 06:15:17.960398 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.960376    5066 scope.go:122] "RemoveContainer" containerID="5f17a1736a61090aa33783f5379c86a15aed4c98e29c57dd219b65937da9ea01"
Oct 30 06:15:17.999945 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:17.999650    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" podStartSLOduration=1.999635375 podStartE2EDuration="1.999635375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:15:17.998894086 +0000 UTC m=+2621.744179419" watchObservedRunningTime="2026-10-30 06:15:17.999635375 +0000 UTC m=+2621.744920706"
Oct 30 06:15:18.115822 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.115744    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:18.118350 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.118198    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 06:15:18.118350 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.118246    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 06:15:18.165722 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.165684    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"]
Oct 30 06:15:18.169194 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.169165    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-815776-29889015-vc78g"]
Oct 30 06:15:18.206300 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.206269    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"]
Oct 30 06:15:18.212274 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.212239    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-db0540-29889015-px5v9"]
Oct 30 06:15:18.405671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.405639    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c" path="/var/lib/kubelet/pods/72b6bd70-d8ba-47d4-acb0-580158cb262c/volumes"
Oct 30 06:15:18.406246 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.406232    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" path="/var/lib/kubelet/pods/8015f2f2-498e-4eb2-92b5-e5cd89b3352a/volumes"
Oct 30 06:15:18.783940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.782158    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 06:15:18.783940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.782221    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" podUID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 06:15:18.982345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.982238    5066 generic.go:356] "Generic (PLEG): container finished" podID="932116d1-8f96-472d-8dd1-5a08f275db18" containerID="1bacdfbc7cfc0e29ae80038c5789aeb84581d31ae91642fa93826bfcd8860fe9" exitCode=60
Oct 30 06:15:18.982903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.982304    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn" event={"ID":"932116d1-8f96-472d-8dd1-5a08f275db18","Type":"ContainerDied","Data":"1bacdfbc7cfc0e29ae80038c5789aeb84581d31ae91642fa93826bfcd8860fe9"}
Oct 30 06:15:18.987646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.985237    5066 scope.go:122] "RemoveContainer" containerID="92d3a805ba76416006fff9fa3aa5eadf7bf984e47afd3fe82f0a72e22941f4ee"
Oct 30 06:15:18.987646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:18.985857    5066 scope.go:122] "RemoveContainer" containerID="1bacdfbc7cfc0e29ae80038c5789aeb84581d31ae91642fa93826bfcd8860fe9"
Oct 30 06:15:18.987646 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:18.986154    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn_odh-ai-gateway-infra(932116d1-8f96-472d-8dd1-5a08f275db18)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn" podUID="932116d1-8f96-472d-8dd1-5a08f275db18"
Oct 30 06:15:19.105581 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.105548    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"]
Oct 30 06:15:19.116145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.116110    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 06:15:19.116312 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.116179    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 06:15:19.389617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.378996    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:19.595604 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.595563    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"
Oct 30 06:15:19.697464 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.696186    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f31f6f0e-e534-48bd-bf2c-9f49feda0561-kube-api-access-4ljk8\" (UniqueName: \"kubernetes.io/projected/f31f6f0e-e534-48bd-bf2c-9f49feda0561-kube-api-access-4ljk8\") pod \"f31f6f0e-e534-48bd-bf2c-9f49feda0561\" (UID: \"f31f6f0e-e534-48bd-bf2c-9f49feda0561\") "
Oct 30 06:15:19.697464 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.696510    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f31f6f0e-e534-48bd-bf2c-9f49feda0561-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f31f6f0e-e534-48bd-bf2c-9f49feda0561-maas-api-service-ca\") pod \"f31f6f0e-e534-48bd-bf2c-9f49feda0561\" (UID: \"f31f6f0e-e534-48bd-bf2c-9f49feda0561\") "
Oct 30 06:15:19.697923 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.697895    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f31f6f0e-e534-48bd-bf2c-9f49feda0561-maas-api-service-ca" pod "f31f6f0e-e534-48bd-bf2c-9f49feda0561" (UID: "f31f6f0e-e534-48bd-bf2c-9f49feda0561"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:15:19.700222 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.700195    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f31f6f0e-e534-48bd-bf2c-9f49feda0561-kube-api-access-4ljk8" pod "f31f6f0e-e534-48bd-bf2c-9f49feda0561" (UID: "f31f6f0e-e534-48bd-bf2c-9f49feda0561"). InnerVolumeSpecName "kube-api-access-4ljk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:15:19.780646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.780098    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 06:15:19.780646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.780178    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" podUID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 06:15:19.798633 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.798582    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f31f6f0e-e534-48bd-bf2c-9f49feda0561-maas-api-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:19.798633 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.798634    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ljk8\" (UniqueName: \"kubernetes.io/projected/f31f6f0e-e534-48bd-bf2c-9f49feda0561-kube-api-access-4ljk8\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:19.999907 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.999803    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"
Oct 30 06:15:19.999907 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.999816    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr" event={"ID":"f31f6f0e-e534-48bd-bf2c-9f49feda0561","Type":"ContainerDied","Data":"841fef2cb27ff951742432abbbc7e34a9a6fdc8ddcefb00f7eecfe1f63232a51"}
Oct 30 06:15:19.999907 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:19.999873    5066 scope.go:122] "RemoveContainer" containerID="c6ea4e7c19ef11db76eed2bd2d11274603e34737e753bf9092ec96d4fbf9fbf3"
Oct 30 06:15:20.085930 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.085881    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"]
Oct 30 06:15:20.114955 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.114919    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"]
Oct 30 06:15:20.117214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.115318    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 06:15:20.117214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.115377    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 06:15:20.122937 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.122901    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-cc33d3-29889015-zwjbr"]
Oct 30 06:15:20.406861 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.406833    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" path="/var/lib/kubelet/pods/f31f6f0e-e534-48bd-bf2c-9f49feda0561/volumes"
Oct 30 06:15:20.513532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.513492    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"
Oct 30 06:15:20.610224 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.610103    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/932116d1-8f96-472d-8dd1-5a08f275db18-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/932116d1-8f96-472d-8dd1-5a08f275db18-maas-api-service-ca\") pod \"932116d1-8f96-472d-8dd1-5a08f275db18\" (UID: \"932116d1-8f96-472d-8dd1-5a08f275db18\") "
Oct 30 06:15:20.610647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.610622    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/932116d1-8f96-472d-8dd1-5a08f275db18-maas-api-service-ca" pod "932116d1-8f96-472d-8dd1-5a08f275db18" (UID: "932116d1-8f96-472d-8dd1-5a08f275db18"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:15:20.610743 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.610700    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/932116d1-8f96-472d-8dd1-5a08f275db18-kube-api-access-6rhpq\" (UniqueName: \"kubernetes.io/projected/932116d1-8f96-472d-8dd1-5a08f275db18-kube-api-access-6rhpq\") pod \"932116d1-8f96-472d-8dd1-5a08f275db18\" (UID: \"932116d1-8f96-472d-8dd1-5a08f275db18\") "
Oct 30 06:15:20.612039 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.612011    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/932116d1-8f96-472d-8dd1-5a08f275db18-maas-api-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:20.615763 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.615737    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/932116d1-8f96-472d-8dd1-5a08f275db18-kube-api-access-6rhpq" pod "932116d1-8f96-472d-8dd1-5a08f275db18" (UID: "932116d1-8f96-472d-8dd1-5a08f275db18"). InnerVolumeSpecName "kube-api-access-6rhpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:15:20.701237 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.700690    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"]
Oct 30 06:15:20.701871 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.701829    5066 cpu_manager.go:414] "containerMap: removing container" podUID="932116d1-8f96-472d-8dd1-5a08f275db18" containerName="cleanup"
Oct 30 06:15:20.701871 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.701861    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="932116d1-8f96-472d-8dd1-5a08f275db18" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.701880    5066 cpu_manager.go:414] "containerMap: removing container" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.701887    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.701899    5066 cpu_manager.go:414] "containerMap: removing container" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.701907    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.701930    5066 cpu_manager.go:414] "containerMap: removing container" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.701936    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.701947    5066 cpu_manager.go:414] "containerMap: removing container" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.701954    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.701986    5066 cpu_manager.go:414] "containerMap: removing container" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.701995    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702030    5066 cpu_manager.go:414] "containerMap: removing container" podUID="932116d1-8f96-472d-8dd1-5a08f275db18" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702038    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="932116d1-8f96-472d-8dd1-5a08f275db18" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702058    5066 cpu_manager.go:414] "containerMap: removing container" podUID="932116d1-8f96-472d-8dd1-5a08f275db18" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702065    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="932116d1-8f96-472d-8dd1-5a08f275db18" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702097    5066 cpu_manager.go:414] "containerMap: removing container" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c" containerName="cleanup"
Oct 30 06:15:20.702124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702105    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c" containerName="cleanup"
Oct 30 06:15:20.702929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702277    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" containerName="cleanup"
Oct 30 06:15:20.702929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702289    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" containerName="cleanup"
Oct 30 06:15:20.702929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702301    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" containerName="cleanup"
Oct 30 06:15:20.702929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702312    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" containerName="cleanup"
Oct 30 06:15:20.702929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702322    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c" containerName="cleanup"
Oct 30 06:15:20.702929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702335    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="932116d1-8f96-472d-8dd1-5a08f275db18" containerName="cleanup"
Oct 30 06:15:20.702929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702344    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" containerName="cleanup"
Oct 30 06:15:20.702929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702354    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="932116d1-8f96-472d-8dd1-5a08f275db18" containerName="cleanup"
Oct 30 06:15:20.702929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702365    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c" containerName="cleanup"
Oct 30 06:15:20.702929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.702373    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c" containerName="cleanup"
Oct 30 06:15:20.708018 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.707985    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"]
Oct 30 06:15:20.708188 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.708103    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.713669 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.713639    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-socket\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.713820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.713677    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-podinfo\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.713820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.713706    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-token\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.713820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.713760    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05d2bec0-3663-4c53-8b93-b0bb50191185-istiod-ca-cert\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.713995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.713856    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wffzd\" (UniqueName: \"kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-kube-api-access-wffzd\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.713995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.713941    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-certs\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.714104 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.714015    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-credential-socket\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.714104 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.714098    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-envoy\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.714215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.714156    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-data\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.714308 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.714275    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rhpq\" (UniqueName: \"kubernetes.io/projected/932116d1-8f96-472d-8dd1-5a08f275db18-kube-api-access-6rhpq\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:15:20.740935 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.740903    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-cc33d3-openshift-default-dockercfg-7lll8\""
Oct 30 06:15:20.776518 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.775060    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 06:15:20.776518 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.775134    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" podUID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 06:15:20.815261 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.815223    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-socket\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.815496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.815279    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-podinfo\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.815496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.815299    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-token\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.815496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.815350    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05d2bec0-3663-4c53-8b93-b0bb50191185-istiod-ca-cert\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.815496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.815435    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wffzd\" (UniqueName: \"kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-kube-api-access-wffzd\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.815782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.815524    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-certs\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.815782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.815616    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-credential-socket\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.815782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.815695    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-envoy\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.815782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.815761    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-data\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.815997 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.815935    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-socket\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.816106 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.816083    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-credential-socket\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.816402 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.816375    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-certs\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.816541 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.816519    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-data\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.816710 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.816690    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05d2bec0-3663-4c53-8b93-b0bb50191185-istiod-ca-cert\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.818339 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.818317    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-podinfo\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.819028 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.819005    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-envoy\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.849926 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.849831    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wffzd\" (UniqueName: \"kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-kube-api-access-wffzd\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:20.868202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:20.862531    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-token\") pod \"e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") " pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:21.011630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.008450    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"
Oct 30 06:15:21.011630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.008478    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn" event={"ID":"932116d1-8f96-472d-8dd1-5a08f275db18","Type":"ContainerDied","Data":"aac5dbf385f2c1d7672cdd551124eaef3cda4efaeebd21bc7c1446caaf1f0e2b"}
Oct 30 06:15:21.011630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.008540    5066 scope.go:122] "RemoveContainer" containerID="1bacdfbc7cfc0e29ae80038c5789aeb84581d31ae91642fa93826bfcd8860fe9"
Oct 30 06:15:21.084626 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.070671    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:21.115469 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.115436    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"]
Oct 30 06:15:21.115780 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.115760    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 06:15:21.115938 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.115800    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 06:15:21.119718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.119659    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-c73fcb-29889015-nd6nn"]
Oct 30 06:15:21.387692 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.387234    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:21.479507 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.479466    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"]
Oct 30 06:15:21.479760 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:15:21.479712    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d2bec0_3663_4c53_8b93_b0bb50191185.slice/crio-df08d2fc83a516dc454044f8a8e95f19ec33f155533f958a581c9361e4585fa1 WatchSource:0}: Error finding container df08d2fc83a516dc454044f8a8e95f19ec33f155533f958a581c9361e4585fa1: Status 404 returned error can't find the container with id df08d2fc83a516dc454044f8a8e95f19ec33f155533f958a581c9361e4585fa1
Oct 30 06:15:21.483357 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.483315    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:15:21.483487 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.483407    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:15:21.483487 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.483447    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:15:21.774478 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.774433    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 06:15:21.774734 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:21.774486    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" podUID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 06:15:22.074775 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:22.074674    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" event={"ID":"05d2bec0-3663-4c53-8b93-b0bb50191185","Type":"ContainerStarted","Data":"33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74"}
Oct 30 06:15:22.074775 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:22.074761    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" event={"ID":"05d2bec0-3663-4c53-8b93-b0bb50191185","Type":"ContainerStarted","Data":"df08d2fc83a516dc454044f8a8e95f19ec33f155533f958a581c9361e4585fa1"}
Oct 30 06:15:22.116263 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:22.116233    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 06:15:22.116480 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:22.116288    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 06:15:22.159615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:22.159095    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" podStartSLOduration=2.159074132 podStartE2EDuration="2.159074132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:15:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:15:22.157437124 +0000 UTC m=+2625.902722453" watchObservedRunningTime="2026-10-30 06:15:22.159074132 +0000 UTC m=+2625.904359464"
Oct 30 06:15:22.403788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:22.403756    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="932116d1-8f96-472d-8dd1-5a08f275db18" path="/var/lib/kubelet/pods/932116d1-8f96-472d-8dd1-5a08f275db18/volumes"
Oct 30 06:15:22.783234 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:22.783143    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:22.784550 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:22.784518    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:22.784910 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:22.784889    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:15:23.076685 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:23.074800    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:23.076685 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:23.074959    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused" start-of-body=
Oct 30 06:15:23.076685 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:23.075007    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" podUID="05d2bec0-3663-4c53-8b93-b0bb50191185" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused"
Oct 30 06:15:23.116234 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:23.116202    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 06:15:23.116420 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:23.116255    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 06:15:23.316614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:23.309792    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:24.072949 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:24.072916    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused" start-of-body=
Oct 30 06:15:24.073153 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:24.072971    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" podUID="05d2bec0-3663-4c53-8b93-b0bb50191185" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused"
Oct 30 06:15:24.115648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:24.115616    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 06:15:24.116030 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:24.115661    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 06:15:25.042613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:25.036837    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:25.071883 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:25.071852    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused" start-of-body=
Oct 30 06:15:25.072046 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:25.071905    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" podUID="05d2bec0-3663-4c53-8b93-b0bb50191185" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused"
Oct 30 06:15:25.121178 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:25.121152    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:25.121690 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:25.121672    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:25.122522 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:25.122501    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:15:26.072221 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:26.072187    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused" start-of-body=
Oct 30 06:15:26.072400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:26.072238    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" podUID="05d2bec0-3663-4c53-8b93-b0bb50191185" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused"
Oct 30 06:15:26.779397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:26.779358    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:27.071933 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:27.071851    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused" start-of-body=
Oct 30 06:15:27.072091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:27.071928    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" podUID="05d2bec0-3663-4c53-8b93-b0bb50191185" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused"
Oct 30 06:15:28.072107 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:28.072073    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused" start-of-body=
Oct 30 06:15:28.072496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:28.072137    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" podUID="05d2bec0-3663-4c53-8b93-b0bb50191185" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused"
Oct 30 06:15:29.092694 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:29.092664    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:29.093811 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:29.093787    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:29.094862 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:29.094844    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:15:29.101150 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:29.100513    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:30.579454 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:30.579420    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:32.065802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:32.065743    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:32.224898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:32.224869    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:32.381409 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:32.381293    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:32.554644 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:15:32.554570    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:15:32.628636 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:32.628605    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:33.567079 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:33.567050    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:33.730880 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:33.730825    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:35.300505 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:35.298279    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:36.830438 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:36.830403    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:38.339928 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:38.336559    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:40.081915 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:40.081882    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:40.241169 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:40.241093    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:41.887394 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:41.887354    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:43.627050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:43.626951    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:45.472097 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:45.472051    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:47.129391 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:47.129114    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:48.322235 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:48.322202    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:48.478567 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:48.478537    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:49.176330 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.176300    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"]
Oct 30 06:15:49.176989 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.176975    5066 cpu_manager.go:414] "containerMap: removing container" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c" containerName="cleanup"
Oct 30 06:15:49.177047 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.176991    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72b6bd70-d8ba-47d4-acb0-580158cb262c" containerName="cleanup"
Oct 30 06:15:49.177047 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.177016    5066 cpu_manager.go:414] "containerMap: removing container" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" containerName="cleanup"
Oct 30 06:15:49.177047 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.177022    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" containerName="cleanup"
Oct 30 06:15:49.177047 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.177038    5066 cpu_manager.go:414] "containerMap: removing container" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" containerName="cleanup"
Oct 30 06:15:49.177047 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.177045    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f31f6f0e-e534-48bd-bf2c-9f49feda0561" containerName="cleanup"
Oct 30 06:15:49.177215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.177156    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="8015f2f2-498e-4eb2-92b5-e5cd89b3352a" containerName="cleanup"
Oct 30 06:15:49.177215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.177168    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="932116d1-8f96-472d-8dd1-5a08f275db18" containerName="cleanup"
Oct 30 06:15:49.182482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.182464    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.205146 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.205105    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-f6302c-serving-cert\""
Oct 30 06:15:49.206817 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.206786    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"]
Oct 30 06:15:49.307920 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.307882    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:49.342783 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.342747    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.343427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.342897    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v9ht\" (UniqueName: \"kubernetes.io/projected/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-kube-api-access-9v9ht\") pod \"maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.343427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.343055    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.343427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.343273    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.445612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.445503    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.445817 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.445634    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9ht\" (UniqueName: \"kubernetes.io/projected/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-kube-api-access-9v9ht\") pod \"maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.445817 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.445737    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.445948 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.445905    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.446905 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.446882    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.449376 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.449345    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.449515 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.449449    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.471849 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.471794    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v9ht\" (UniqueName: \"kubernetes.io/projected/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-kube-api-access-9v9ht\") pod \"maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.475333 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.475307    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:49.498879 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.498853    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:49.807720 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:49.807691    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"]
Oct 30 06:15:49.808606 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:15:49.808568    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9c9408_f6b3_4b64_8238_4d8dcbd11dbf.slice/crio-18348fc529cdbaa83d7d3a83573108fd3504c098ea8d150b1c9e70ca382e8612 WatchSource:0}: Error finding container 18348fc529cdbaa83d7d3a83573108fd3504c098ea8d150b1c9e70ca382e8612: Status 404 returned error can't find the container with id 18348fc529cdbaa83d7d3a83573108fd3504c098ea8d150b1c9e70ca382e8612
Oct 30 06:15:50.328357 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:50.328321    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj" event={"ID":"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf","Type":"ContainerStarted","Data":"621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86"}
Oct 30 06:15:50.328570 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:50.328377    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj" event={"ID":"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf","Type":"ContainerStarted","Data":"18348fc529cdbaa83d7d3a83573108fd3504c098ea8d150b1c9e70ca382e8612"}
Oct 30 06:15:50.328570 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:50.328441    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:50.381619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:50.381474    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj" podStartSLOduration=1.056363637 podStartE2EDuration="1.381455531s" totalImagesPullingTime="325.091894ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:15:50.378262119 +0000 UTC m=+2654.123547452" watchObservedRunningTime="2026-10-30 06:15:50.381455531 +0000 UTC m=+2654.126740864"
Oct 30 06:15:51.247645 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:51.247552    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:52.834071 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:52.833993    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:54.345332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:54.345295    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:56.140078 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:56.140028    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:56.341059 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:56.341031    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:15:57.696663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:57.696577    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:15:59.425264 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:15:59.425226    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:01.041659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:01.041627    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:01.940429 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:01.940395    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"]
Oct 30 06:16:01.946208 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:01.946185    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:01.982190 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:01.982151    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-24db22-openshift-default-dockercfg-75mbl\""
Oct 30 06:16:01.988308 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:01.988274    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"]
Oct 30 06:16:02.058392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.058352    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsrvs\" (UniqueName: \"kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-kube-api-access-bsrvs\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.058917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.058485    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-podinfo\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.058917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.058536    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-data\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.058917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.058559    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-token\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.058917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.058716    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-credential-socket\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.058917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.058762    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-socket\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.058917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.058837    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f575bda0-bead-41dc-a5e5-d633cad8e59d-istiod-ca-cert\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.059229 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.058979    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-envoy\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.059229 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.059048    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-certs\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.160576 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.160540    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-podinfo\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.160791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.160622    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-data\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.160791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.160642    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-token\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.160791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.160679    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-credential-socket\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.160791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.160720    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-socket\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.160791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.160770    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f575bda0-bead-41dc-a5e5-d633cad8e59d-istiod-ca-cert\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.161148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.160850    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-envoy\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.161148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.160903    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-certs\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.161148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.160967    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsrvs\" (UniqueName: \"kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-kube-api-access-bsrvs\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.161148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.161108    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-data\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.161380 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.161302    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-socket\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.161380 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.161355    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-certs\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.161510 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.161398    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-credential-socket\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.161671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.161646    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f575bda0-bead-41dc-a5e5-d633cad8e59d-istiod-ca-cert\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.163173 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.163152    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-envoy\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.163265 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.163217    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-podinfo\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.187522 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.187496    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-token\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.187695 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.187562    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsrvs\" (UniqueName: \"kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-kube-api-access-bsrvs\") pod \"e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") " pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.262465 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.262378    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:02.553936 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:16:02.553849    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:16:02.618718 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:16:02.618686    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf575bda0_bead_41dc_a5e5_d633cad8e59d.slice/crio-b440dcc5c5c96ba0d4c95c4df897864c410defdf3527a54a7d6615fa962e8b4d WatchSource:0}: Error finding container b440dcc5c5c96ba0d4c95c4df897864c410defdf3527a54a7d6615fa962e8b4d: Status 404 returned error can't find the container with id b440dcc5c5c96ba0d4c95c4df897864c410defdf3527a54a7d6615fa962e8b4d
Oct 30 06:16:02.620057 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.620028    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"]
Oct 30 06:16:02.622116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.622088    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:16:02.622207 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.622162    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:16:02.622207 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.622191    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:16:02.856104 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:02.855752    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:03.370756 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:03.370725    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:03.467168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:03.467131    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986" event={"ID":"f575bda0-bead-41dc-a5e5-d633cad8e59d","Type":"ContainerStarted","Data":"c52bc9b8a2fca45f7cb8b8781d9d02e4f860e6b2d95c3aeb00831d57924f7c74"}
Oct 30 06:16:03.467467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:03.467195    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986" event={"ID":"f575bda0-bead-41dc-a5e5-d633cad8e59d","Type":"ContainerStarted","Data":"b440dcc5c5c96ba0d4c95c4df897864c410defdf3527a54a7d6615fa962e8b4d"}
Oct 30 06:16:03.517419 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:03.517366    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986" podStartSLOduration=2.517340098 podStartE2EDuration="2.517340098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:16:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:16:03.516074438 +0000 UTC m=+2667.261359821" watchObservedRunningTime="2026-10-30 06:16:03.517340098 +0000 UTC m=+2667.262625429"
Oct 30 06:16:03.539535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:03.539488    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:03.712497 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:03.712464    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:04.263131 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:04.263104    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:04.264744 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:04.264719    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.169:15021/healthz/ready\": dial tcp 10.132.0.169:15021: connect: connection refused" start-of-body=
Oct 30 06:16:04.264890 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:04.264766    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986" podUID="f575bda0-bead-41dc-a5e5-d633cad8e59d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.169:15021/healthz/ready\": dial tcp 10.132.0.169:15021: connect: connection refused"
Oct 30 06:16:04.748614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:04.748418    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:05.262676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:05.262645    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.169:15021/healthz/ready\": dial tcp 10.132.0.169:15021: connect: connection refused" start-of-body=
Oct 30 06:16:05.262853 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:05.262693    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986" podUID="f575bda0-bead-41dc-a5e5-d633cad8e59d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.169:15021/healthz/ready\": dial tcp 10.132.0.169:15021: connect: connection refused"
Oct 30 06:16:05.593127 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:05.593037    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:05.965944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:05.965902    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:06.161694 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:06.161644    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:06.294577 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:06.294494    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.169:15021/healthz/ready\": dial tcp 10.132.0.169:15021: connect: connection refused" start-of-body=
Oct 30 06:16:06.294877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:06.294831    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986" podUID="f575bda0-bead-41dc-a5e5-d633cad8e59d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.169:15021/healthz/ready\": dial tcp 10.132.0.169:15021: connect: connection refused"
Oct 30 06:16:06.384083 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:06.383986    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:06.760949 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:06.760916    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:06.936614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:06.935988    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:07.262942 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:07.262898    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.169:15021/healthz/ready\": dial tcp 10.132.0.169:15021: connect: connection refused" start-of-body=
Oct 30 06:16:07.263366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:07.262963    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986" podUID="f575bda0-bead-41dc-a5e5-d633cad8e59d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.169:15021/healthz/ready\": dial tcp 10.132.0.169:15021: connect: connection refused"
Oct 30 06:16:08.262940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:08.262903    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.169:15021/healthz/ready\": dial tcp 10.132.0.169:15021: connect: connection refused" start-of-body=
Oct 30 06:16:08.263445 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:08.262965    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986" podUID="f575bda0-bead-41dc-a5e5-d633cad8e59d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.169:15021/healthz/ready\": dial tcp 10.132.0.169:15021: connect: connection refused"
Oct 30 06:16:08.678264 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:08.677797    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:09.263776 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:09.263743    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.169:15021/healthz/ready\": dial tcp 10.132.0.169:15021: connect: connection refused" start-of-body=
Oct 30 06:16:09.264302 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:09.263801    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986" podUID="f575bda0-bead-41dc-a5e5-d633cad8e59d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.169:15021/healthz/ready\": dial tcp 10.132.0.169:15021: connect: connection refused"
Oct 30 06:16:09.987403 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:09.987365    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:10.151756 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:10.151720    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:10.269042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:10.268966    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:10.269500 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:10.269451    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:10.270212 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:10.270193    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:16:11.040023 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:11.039987    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:11.732602 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:11.732562    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:12.041809 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:12.041728    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:12.571099 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:12.571064    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:13.263917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:13.263885    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:13.453558 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:13.453528    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:15.288309 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:15.288275    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:17.665273 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:17.658113    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:19.944800 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:19.944769    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:21.846241 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:21.846210    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:23.647903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:23.647866    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:24.661482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:24.661430    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:24.829132 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:24.829074    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:25.016218 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:25.016112    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:26.035205 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:26.035175    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:27.772474 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:27.772423    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:29.527525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:29.526939    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:31.345311 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:31.345197    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:32.554110 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:16:32.554076    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:16:32.750223 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:32.750192    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:33.291976 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:33.291928    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:33.690533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:33.690486    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:35.136251 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:35.136010    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:36.535568 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:36.535538    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:36.702689 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:36.702661    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:36.843257 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:36.833896    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:16:36.878535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:36.878504    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:16:36.954288 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:36.954254    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:38.996618 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:38.995764    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:41.235349 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:41.235314    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:43.287419 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:43.287384    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:45.042036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:45.042000    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:45.312670 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:45.310627    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:46.608613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:46.608487    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:47.456194 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:47.456160    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:49.419413 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:49.419348    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:50.751448 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:50.751414    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:50.924070 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:50.924037    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:51.954410 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:51.954335    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:52.127687 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:52.127651    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:53.973769 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:53.973573    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:55.741222 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:55.741161    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:57.409075 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:57.409045    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:16:59.242222 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:16:59.242178    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:00.927941 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:00.927906    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:02.554472 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:17:02.554434    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:17:02.690834 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:02.690803    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:02.866005 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:02.865896    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:03.046669 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:03.046636    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:04.843229 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:04.843196    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:06.634612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:06.633153    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:08.543168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:08.541332    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:10.440684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:10.440618    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:12.668278 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:12.667782    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:13.398184 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:13.398147    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:15.223370 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:15.223331    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:17.002292 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:17.002200    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:18.369367 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:18.369313    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:18.546827 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:18.546799    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:19.653217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:19.652424    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:21.465303 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:21.465268    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:23.274968 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:23.274934    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:25.146436 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:25.146382    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:26.983615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:26.978921    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:28.365831 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:28.365789    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:28.563618 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:28.559046    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:28.894231 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:28.894191    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"]
Oct 30 06:17:28.899840 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:28.899813    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:28.921741 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:28.921713    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"]
Oct 30 06:17:29.007934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.007903    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-0f980e-openshift-default-dockercfg-sk7nx\""
Oct 30 06:17:29.049656 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.049605    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istiod-ca-cert\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.049845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.049699    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-credential-socket\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.049845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.049770    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-token\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.049845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.049798    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-data\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.049994 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.049973    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-certs\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.050053 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.050040    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vchc\" (UniqueName: \"kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-kube-api-access-7vchc\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.050150 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.050127    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-podinfo\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.050215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.050172    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-socket\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.050327 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.050312    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-envoy\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152114 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152032    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-certs\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152114 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152097    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vchc\" (UniqueName: \"kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-kube-api-access-7vchc\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152359 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152123    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-podinfo\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152359 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152143    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-socket\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152359 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152220    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-envoy\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152359 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152274    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istiod-ca-cert\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152359 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152295    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-credential-socket\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152667 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152368    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-token\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152667 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152404    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-data\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152667 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152508    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-certs\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152667 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152511    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-socket\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152867 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152762    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-credential-socket\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.152867 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.152820    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-data\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.153294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.153275    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istiod-ca-cert\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.154863 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.154840    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-envoy\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.154964 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.154846    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-podinfo\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.177652 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.177576    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vchc\" (UniqueName: \"kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-kube-api-access-7vchc\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.178044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.178026    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-token\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.218537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.218507    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:29.549474 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.549447    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"]
Oct 30 06:17:29.549867 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:17:29.549552    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd717f7_573f_4bc8_99b6_31ccdaa6d45d.slice/crio-08969bf3a31af920a60299a9390dbdec57ac029d70986dd8fbd2e0a4ae4d950a WatchSource:0}: Error finding container 08969bf3a31af920a60299a9390dbdec57ac029d70986dd8fbd2e0a4ae4d950a: Status 404 returned error can't find the container with id 08969bf3a31af920a60299a9390dbdec57ac029d70986dd8fbd2e0a4ae4d950a
Oct 30 06:17:29.552583 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.552553    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:17:29.552688 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.552642    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:17:29.552688 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.552675    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:17:29.644369 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:29.643536    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:30.274912 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:30.274876    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c" event={"ID":"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d","Type":"ContainerStarted","Data":"1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5"}
Oct 30 06:17:30.275111 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:30.274944    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c" event={"ID":"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d","Type":"ContainerStarted","Data":"08969bf3a31af920a60299a9390dbdec57ac029d70986dd8fbd2e0a4ae4d950a"}
Oct 30 06:17:30.347614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:30.347162    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c" podStartSLOduration=2.347127553 podStartE2EDuration="2.347127553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:17:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:17:30.345210232 +0000 UTC m=+2754.090495565" watchObservedRunningTime="2026-10-30 06:17:30.347127553 +0000 UTC m=+2754.092412885"
Oct 30 06:17:31.218859 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:31.218824    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:31.220807 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:31.220780    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused" start-of-body=
Oct 30 06:17:31.220940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:31.220832    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c" podUID="9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused"
Oct 30 06:17:31.550306 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:31.550218    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:32.218888 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:32.218855    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused" start-of-body=
Oct 30 06:17:32.219330 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:32.218914    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c" podUID="9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused"
Oct 30 06:17:32.577036 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:17:32.576945    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:17:32.771857 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:32.771825    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:32.949239 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:32.949202    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:33.219357 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:33.219270    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused" start-of-body=
Oct 30 06:17:33.219956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:33.219920    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c" podUID="9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused"
Oct 30 06:17:33.264717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:33.264684    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:33.472804 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:33.458536    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:33.797670 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:33.790415    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:33.955427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:33.955381    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:34.219210 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:34.219170    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused" start-of-body=
Oct 30 06:17:34.219403 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:34.219238    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c" podUID="9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused"
Oct 30 06:17:35.219275 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:35.219234    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused" start-of-body=
Oct 30 06:17:35.219458 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:35.219284    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c" podUID="9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused"
Oct 30 06:17:35.816737 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:35.816698    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:36.381932 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:36.381900    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:36.383048 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:36.382989    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:36.385622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:36.385582    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:17:37.226393 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:37.226360    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:37.431306 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:37.431261    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:38.829818 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:38.829777    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:39.098029 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:39.097923    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:39.328619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:39.318188    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:40.754354 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:40.754309    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:42.548555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:42.548520    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:43.748094 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:43.748060    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:44.639954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:44.639916    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:44.810287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:44.810258    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:45.377523 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:45.377487    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:45.580728 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:45.580696    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:46.658165 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:46.658132    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:47.408943 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:47.408909    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:49.314058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:49.314026    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:49.546741 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:49.546535    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:50.719526 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:50.719495    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:51.464873 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:51.464844    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:51.840777 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:51.840683    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:52.094464 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:52.072043    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:53.543573 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:53.543538    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:55.301067 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:55.301031    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:55.911010 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:55.910971    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:56.932098 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:56.932065    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"]
Oct 30 06:17:56.932551 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:56.932279    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy" containerID="cri-o://583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9" gracePeriod=30
Oct 30 06:17:57.351738 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:57.346981    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:17:59.150217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:17:59.150135    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:00.922532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:00.922494    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:01.614835 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:01.614795    5066 patch_prober.go:29] interesting pod/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:18:01.615016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:01.614851    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:18:02.229417 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.229392    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 06:18:02.253939 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.253909    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-token\" (UniqueName: \"kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-token\") pod \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") "
Oct 30 06:18:02.254127 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254010    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-kube-api-access-x6tkn\" (UniqueName: \"kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-kube-api-access-x6tkn\") pod \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") "
Oct 30 06:18:02.254127 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254049    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-certs\") pod \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") "
Oct 30 06:18:02.254127 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254081    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-socket\") pod \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") "
Oct 30 06:18:02.254127 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254125    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istiod-ca-cert\") pod \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") "
Oct 30 06:18:02.254371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254180    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-credential-socket\") pod \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") "
Oct 30 06:18:02.254371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254230    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-envoy\") pod \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") "
Oct 30 06:18:02.254371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254312    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-data\") pod \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") "
Oct 30 06:18:02.254371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254349    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-podinfo\") pod \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\" (UID: \"61a0a834-7508-4682-bd91-d4cde3cbd2ad\") "
Oct 30 06:18:02.254643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254375    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-certs" pod "61a0a834-7508-4682-bd91-d4cde3cbd2ad" (UID: "61a0a834-7508-4682-bd91-d4cde3cbd2ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:18:02.254643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254493    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-socket" pod "61a0a834-7508-4682-bd91-d4cde3cbd2ad" (UID: "61a0a834-7508-4682-bd91-d4cde3cbd2ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:18:02.254643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254545    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-credential-socket" pod "61a0a834-7508-4682-bd91-d4cde3cbd2ad" (UID: "61a0a834-7508-4682-bd91-d4cde3cbd2ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:18:02.254985 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254728    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-data" pod "61a0a834-7508-4682-bd91-d4cde3cbd2ad" (UID: "61a0a834-7508-4682-bd91-d4cde3cbd2ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:18:02.254985 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.254631    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istiod-ca-cert" pod "61a0a834-7508-4682-bd91-d4cde3cbd2ad" (UID: "61a0a834-7508-4682-bd91-d4cde3cbd2ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:18:02.255182 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.255160    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:02.255255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.255189    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:02.255255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.255203    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:02.255255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.255218    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:02.255255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.255231    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:02.256437 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.256415    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-token" pod "61a0a834-7508-4682-bd91-d4cde3cbd2ad" (UID: "61a0a834-7508-4682-bd91-d4cde3cbd2ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:18:02.256572 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.256525    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-kube-api-access-x6tkn" pod "61a0a834-7508-4682-bd91-d4cde3cbd2ad" (UID: "61a0a834-7508-4682-bd91-d4cde3cbd2ad"). InnerVolumeSpecName "kube-api-access-x6tkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:18:02.256814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.256797    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-podinfo" pod "61a0a834-7508-4682-bd91-d4cde3cbd2ad" (UID: "61a0a834-7508-4682-bd91-d4cde3cbd2ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:18:02.256904 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.256892    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-envoy" pod "61a0a834-7508-4682-bd91-d4cde3cbd2ad" (UID: "61a0a834-7508-4682-bd91-d4cde3cbd2ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:18:02.356760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.356726    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:02.356760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.356757    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:02.356760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.356765    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:02.357006 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.356774    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6tkn\" (UniqueName: \"kubernetes.io/projected/61a0a834-7508-4682-bd91-d4cde3cbd2ad-kube-api-access-x6tkn\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:02.632538 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.632412    5066 generic.go:356] "Generic (PLEG): container finished" podID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerID="583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9" exitCode=0
Oct 30 06:18:02.632538 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.632457    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" event={"ID":"61a0a834-7508-4682-bd91-d4cde3cbd2ad","Type":"ContainerDied","Data":"583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9"}
Oct 30 06:18:02.632538 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.632476    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"
Oct 30 06:18:02.632538 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.632522    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d" event={"ID":"61a0a834-7508-4682-bd91-d4cde3cbd2ad","Type":"ContainerDied","Data":"4a25b4e660c7c91078e251b3e0fad152bef19719d4695037cad748b51871c7dc"}
Oct 30 06:18:02.632888 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.632549    5066 scope.go:122] "RemoveContainer" containerID="583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9"
Oct 30 06:18:02.632888 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:18:02.632761    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:18:02.645328 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.645301    5066 scope.go:122] "RemoveContainer" containerID="583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9"
Oct 30 06:18:02.645991 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:18:02.645936    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9\": container with ID starting with 583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9 not found: ID does not exist" containerID="583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9"
Oct 30 06:18:02.646162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.645985    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9"} err="failed to get container status \"583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9\": rpc error: code = NotFound desc = could not find container \"583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9\": container with ID starting with 583987296c9d6a481b120d87ca1ba6c0e98cfa92411940036e072777d25559c9 not found: ID does not exist"
Oct 30 06:18:02.696276 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.696239    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"]
Oct 30 06:18:02.702123 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.702089    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-1354a5-openshift-default-6c46966c87-5cb2d"]
Oct 30 06:18:02.947348 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:02.947314    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:03.731839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.731809    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"]
Oct 30 06:18:03.732511 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.732494    5066 cpu_manager.go:414] "containerMap: removing container" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy"
Oct 30 06:18:03.732511 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.732510    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy"
Oct 30 06:18:03.732703 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.732666    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" containerName="istio-proxy"
Oct 30 06:18:03.736192 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.736171    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.762031 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.761995    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"]
Oct 30 06:18:03.771095 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.771063    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w1-0f980e\"/\"openshift-service-ca.crt\""
Oct 30 06:18:03.771315 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.771082    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w1-0f980e\"/\"kube-root-ca.crt\""
Oct 30 06:18:03.771315 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.771148    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w1-0f980e\"/\"default-dockercfg-wvlht\""
Oct 30 06:18:03.771315 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.771178    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w1-0f980e\"/\"rate-test-model-0f980e-kserve-self-signed-certs\""
Oct 30 06:18:03.780342 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.780311    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-model-cache\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.780508 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.780357    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdnzt\" (UniqueName: \"kubernetes.io/projected/8e98b564-3662-481a-acc0-125897de5903-kube-api-access-qdnzt\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.780569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.780502    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-dshm\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.780924 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.780668    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-tmp-dir\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.780924 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.780719    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-home\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.780924 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.780772    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e98b564-3662-481a-acc0-125897de5903-tls-certs\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.832252 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.832213    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:03.882746 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.882711    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-tmp-dir\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.882923 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.882762    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-home\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.882923 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.882803    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e98b564-3662-481a-acc0-125897de5903-tls-certs\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.883084 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.883005    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-model-cache\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.883084 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.883036    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdnzt\" (UniqueName: \"kubernetes.io/projected/8e98b564-3662-481a-acc0-125897de5903-kube-api-access-qdnzt\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.883199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.883109    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-dshm\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.883199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.883183    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-home\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.883316 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.883199    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-tmp-dir\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.883420 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.883392    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-model-cache\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.885864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.885837    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-dshm\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.885965 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.885951    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e98b564-3662-481a-acc0-125897de5903-tls-certs\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:03.913439 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:03.913376    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdnzt\" (UniqueName: \"kubernetes.io/projected/8e98b564-3662-481a-acc0-125897de5903-kube-api-access-qdnzt\") pod \"rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") " pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:04.052963 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:04.052873    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:04.406024 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:04.405993    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61a0a834-7508-4682-bd91-d4cde3cbd2ad" path="/var/lib/kubelet/pods/61a0a834-7508-4682-bd91-d4cde3cbd2ad/volumes"
Oct 30 06:18:04.495229 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:04.495195    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"]
Oct 30 06:18:04.500248 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:18:04.500209    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e98b564_3662_481a_acc0_125897de5903.slice/crio-9dc0b24e6cd4eadd36378aa5729e594daf7ee6d702b21e90dcfacd8df0273d70 WatchSource:0}: Error finding container 9dc0b24e6cd4eadd36378aa5729e594daf7ee6d702b21e90dcfacd8df0273d70: Status 404 returned error can't find the container with id 9dc0b24e6cd4eadd36378aa5729e594daf7ee6d702b21e90dcfacd8df0273d70
Oct 30 06:18:04.651031 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:04.650997    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b" event={"ID":"8e98b564-3662-481a-acc0-125897de5903","Type":"ContainerStarted","Data":"516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af"}
Oct 30 06:18:04.651194 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:04.651038    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b" event={"ID":"8e98b564-3662-481a-acc0-125897de5903","Type":"ContainerStarted","Data":"9dc0b24e6cd4eadd36378aa5729e594daf7ee6d702b21e90dcfacd8df0273d70"}
Oct 30 06:18:04.699928 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:04.699810    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b" podStartSLOduration=1.6997951119999999 podStartE2EDuration="1.699795112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:18:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:18:04.697372708 +0000 UTC m=+2788.442658042" watchObservedRunningTime="2026-10-30 06:18:04.699795112 +0000 UTC m=+2788.445080475"
Oct 30 06:18:04.973980 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:04.973892    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:05.056695 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:05.056659    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:06.895884 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:06.891105    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:07.982686 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:07.982647    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:08.145022 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.144988    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:08.331932 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.331840    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:08.364142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.364109    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"]
Oct 30 06:18:08.364432 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.364407    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8" podUID="44d4c399-5239-4ac4-b4f8-a15ecf2313fc" containerName="maas-api" containerID="cri-o://609d32e82d5220b155efb5e07722aaec44456552ac2e92b047958b30e0482eb3" gracePeriod=30
Oct 30 06:18:08.560634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.557925    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:08.572654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.569569    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"]
Oct 30 06:18:08.572654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.569822    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj" podUID="16484b77-48d2-4522-a074-1d814552fb11" containerName="payload-processing" containerID="cri-o://d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad" gracePeriod=30
Oct 30 06:18:08.605798 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.605767    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"]
Oct 30 06:18:08.606254 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.606221    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj" podUID="b360e5eb-1915-47c0-9f8f-7adabbb6f52f" containerName="payload-pre-processing" containerID="cri-o://2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f" gracePeriod=30
Oct 30 06:18:08.699250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.699070    5066 generic.go:356] "Generic (PLEG): container finished" podID="44d4c399-5239-4ac4-b4f8-a15ecf2313fc" containerID="609d32e82d5220b155efb5e07722aaec44456552ac2e92b047958b30e0482eb3" exitCode=0
Oct 30 06:18:08.699250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.699220    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8" event={"ID":"44d4c399-5239-4ac4-b4f8-a15ecf2313fc","Type":"ContainerDied","Data":"609d32e82d5220b155efb5e07722aaec44456552ac2e92b047958b30e0482eb3"}
Oct 30 06:18:08.733049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.732959    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:08.771665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.768851    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:18:08.898177 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.898078    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:08.923788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.922354    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-metrics-certs\") pod \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") "
Oct 30 06:18:08.923788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.922517    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-maas-api-tls\") pod \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") "
Oct 30 06:18:08.923788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.922742    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-openshift-service-ca\") pod \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") "
Oct 30 06:18:08.923788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.922839    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-kube-api-access-j7vt2\" (UniqueName: \"kubernetes.io/projected/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-kube-api-access-j7vt2\") pod \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\" (UID: \"44d4c399-5239-4ac4-b4f8-a15ecf2313fc\") "
Oct 30 06:18:08.926046 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:08.925377    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-openshift-service-ca" pod "44d4c399-5239-4ac4-b4f8-a15ecf2313fc" (UID: "44d4c399-5239-4ac4-b4f8-a15ecf2313fc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:18:09.004808 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.004516    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-kube-api-access-j7vt2" pod "44d4c399-5239-4ac4-b4f8-a15ecf2313fc" (UID: "44d4c399-5239-4ac4-b4f8-a15ecf2313fc"). InnerVolumeSpecName "kube-api-access-j7vt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:18:09.037658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.013680    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-metrics-certs" pod "44d4c399-5239-4ac4-b4f8-a15ecf2313fc" (UID: "44d4c399-5239-4ac4-b4f8-a15ecf2313fc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:18:09.037658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.025221    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:09.037658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.025248    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:09.037658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.025265    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7vt2\" (UniqueName: \"kubernetes.io/projected/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-kube-api-access-j7vt2\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:09.093614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.090077    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-maas-api-tls" pod "44d4c399-5239-4ac4-b4f8-a15ecf2313fc" (UID: "44d4c399-5239-4ac4-b4f8-a15ecf2313fc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:18:09.114611 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.113044    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"
Oct 30 06:18:09.135622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.127097    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44d4c399-5239-4ac4-b4f8-a15ecf2313fc-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:09.211942 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.211906    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"
Oct 30 06:18:09.230123 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.230078    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16484b77-48d2-4522-a074-1d814552fb11-kube-api-access-g2wpf\" (UniqueName: \"kubernetes.io/projected/16484b77-48d2-4522-a074-1d814552fb11-kube-api-access-g2wpf\") pod \"16484b77-48d2-4522-a074-1d814552fb11\" (UID: \"16484b77-48d2-4522-a074-1d814552fb11\") "
Oct 30 06:18:09.230427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.230332    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16484b77-48d2-4522-a074-1d814552fb11-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16484b77-48d2-4522-a074-1d814552fb11-plugins-config-volume\") pod \"16484b77-48d2-4522-a074-1d814552fb11\" (UID: \"16484b77-48d2-4522-a074-1d814552fb11\") "
Oct 30 06:18:09.231179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.231110    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16484b77-48d2-4522-a074-1d814552fb11-plugins-config-volume" pod "16484b77-48d2-4522-a074-1d814552fb11" (UID: "16484b77-48d2-4522-a074-1d814552fb11"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:18:09.233179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.233156    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16484b77-48d2-4522-a074-1d814552fb11-kube-api-access-g2wpf" pod "16484b77-48d2-4522-a074-1d814552fb11" (UID: "16484b77-48d2-4522-a074-1d814552fb11"). InnerVolumeSpecName "kube-api-access-g2wpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:18:09.331400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.331366    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-plugins-config-volume\") pod \"b360e5eb-1915-47c0-9f8f-7adabbb6f52f\" (UID: \"b360e5eb-1915-47c0-9f8f-7adabbb6f52f\") "
Oct 30 06:18:09.331691 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.331670    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-kube-api-access-rkvw8\" (UniqueName: \"kubernetes.io/projected/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-kube-api-access-rkvw8\") pod \"b360e5eb-1915-47c0-9f8f-7adabbb6f52f\" (UID: \"b360e5eb-1915-47c0-9f8f-7adabbb6f52f\") "
Oct 30 06:18:09.331808 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.331783    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-plugins-config-volume" pod "b360e5eb-1915-47c0-9f8f-7adabbb6f52f" (UID: "b360e5eb-1915-47c0-9f8f-7adabbb6f52f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:18:09.332385 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.332360    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2wpf\" (UniqueName: \"kubernetes.io/projected/16484b77-48d2-4522-a074-1d814552fb11-kube-api-access-g2wpf\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:09.332497 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.332387    5066 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-plugins-config-volume\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:09.332497 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.332402    5066 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16484b77-48d2-4522-a074-1d814552fb11-plugins-config-volume\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:09.334229 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.334202    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-kube-api-access-rkvw8" pod "b360e5eb-1915-47c0-9f8f-7adabbb6f52f" (UID: "b360e5eb-1915-47c0-9f8f-7adabbb6f52f"). InnerVolumeSpecName "kube-api-access-rkvw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:18:09.414349 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.414317    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:09.434223 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.434147    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkvw8\" (UniqueName: \"kubernetes.io/projected/b360e5eb-1915-47c0-9f8f-7adabbb6f52f-kube-api-access-rkvw8\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:09.629702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.629669    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:09.707755 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.707665    5066 generic.go:356] "Generic (PLEG): container finished" podID="b360e5eb-1915-47c0-9f8f-7adabbb6f52f" containerID="2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f" exitCode=0
Oct 30 06:18:09.707755 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.707711    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"
Oct 30 06:18:09.707755 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.707743    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj" event={"ID":"b360e5eb-1915-47c0-9f8f-7adabbb6f52f","Type":"ContainerDied","Data":"2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f"}
Oct 30 06:18:09.708015 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.707799    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj" event={"ID":"b360e5eb-1915-47c0-9f8f-7adabbb6f52f","Type":"ContainerDied","Data":"f649242ac6c29632683523089a9a57c1402dba687311f7a7d071a022b93c935c"}
Oct 30 06:18:09.708015 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.707822    5066 scope.go:122] "RemoveContainer" containerID="2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f"
Oct 30 06:18:09.709510 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.709463    5066 generic.go:356] "Generic (PLEG): container finished" podID="16484b77-48d2-4522-a074-1d814552fb11" containerID="d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad" exitCode=0
Oct 30 06:18:09.709688 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.709553    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj" event={"ID":"16484b77-48d2-4522-a074-1d814552fb11","Type":"ContainerDied","Data":"d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad"}
Oct 30 06:18:09.709688 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.709619    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj" event={"ID":"16484b77-48d2-4522-a074-1d814552fb11","Type":"ContainerDied","Data":"e3e58126d95e087728a58ee16cbc7fc4908b23eab5df7631834571fe15b3b706"}
Oct 30 06:18:09.709796 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.709627    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"
Oct 30 06:18:09.711264 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.711236    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"
Oct 30 06:18:09.711264 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.711241    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8" event={"ID":"44d4c399-5239-4ac4-b4f8-a15ecf2313fc","Type":"ContainerDied","Data":"2eddd2c49c34624487f502f9af7b8b68a733479fb8489506aeaf9e8cf2f9dd8e"}
Oct 30 06:18:09.723001 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.722966    5066 scope.go:122] "RemoveContainer" containerID="2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f"
Oct 30 06:18:09.723319 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:18:09.723296    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f\": container with ID starting with 2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f not found: ID does not exist" containerID="2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f"
Oct 30 06:18:09.723395 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.723334    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f"} err="failed to get container status \"2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f\": rpc error: code = NotFound desc = could not find container \"2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f\": container with ID starting with 2c8b0c79e5718f8c66fdd69b20a1a09754f7c4904b2073eff19289edb431101f not found: ID does not exist"
Oct 30 06:18:09.723395 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.723358    5066 scope.go:122] "RemoveContainer" containerID="d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad"
Oct 30 06:18:09.735344 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.735325    5066 scope.go:122] "RemoveContainer" containerID="d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad"
Oct 30 06:18:09.735680 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:18:09.735657    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad\": container with ID starting with d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad not found: ID does not exist" containerID="d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad"
Oct 30 06:18:09.735751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.735688    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad"} err="failed to get container status \"d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad\": rpc error: code = NotFound desc = could not find container \"d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad\": container with ID starting with d410a5b739337b75fa1eaeb7c28feb3a3bd4d4112176e1b90a50e3b2c8756bad not found: ID does not exist"
Oct 30 06:18:09.735751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.735704    5066 scope.go:122] "RemoveContainer" containerID="609d32e82d5220b155efb5e07722aaec44456552ac2e92b047958b30e0482eb3"
Oct 30 06:18:09.769795 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.769762    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"]
Oct 30 06:18:09.773518 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.773491    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-db0540-786d5c8cd7-rzbxj"]
Oct 30 06:18:09.821256 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.821224    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"]
Oct 30 06:18:09.827548 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.827523    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-db0540-85875d5ff6-hgxhj"]
Oct 30 06:18:09.886003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.885962    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"]
Oct 30 06:18:09.889743 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:09.889723    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-db0540-f7b489f56-mmjb8"]
Oct 30 06:18:10.103282 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:10.103170    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:10.404154 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:10.404127    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16484b77-48d2-4522-a074-1d814552fb11" path="/var/lib/kubelet/pods/16484b77-48d2-4522-a074-1d814552fb11/volumes"
Oct 30 06:18:10.404683 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:10.404668    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44d4c399-5239-4ac4-b4f8-a15ecf2313fc" path="/var/lib/kubelet/pods/44d4c399-5239-4ac4-b4f8-a15ecf2313fc/volumes"
Oct 30 06:18:10.405057 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:10.405045    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b360e5eb-1915-47c0-9f8f-7adabbb6f52f" path="/var/lib/kubelet/pods/b360e5eb-1915-47c0-9f8f-7adabbb6f52f/volumes"
Oct 30 06:18:11.373198 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:11.373087    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:13.201304 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:13.199563    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:14.054338 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:14.054306    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:14.067141 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:14.067107    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:14.761569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:14.761538    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:18:15.625878 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:15.625852    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:16.664909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:16.664870    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:16.839117 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:16.839086    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:17.019123 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:17.019038    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:17.550253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:17.550222    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:19.549455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:19.549411    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:21.430258 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:21.429811    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:23.475626 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:23.475522    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:26.042205 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:26.042167    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:27.757687 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:27.745385    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:29.436211 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:29.436139    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:31.446193 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:31.446157    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:32.553437 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:18:32.553396    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:18:33.024120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.024087    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:33.641469 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.641433    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"]
Oct 30 06:18:33.642124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.642109    5066 cpu_manager.go:414] "containerMap: removing container" podUID="16484b77-48d2-4522-a074-1d814552fb11" containerName="payload-processing"
Oct 30 06:18:33.642177 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.642126    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16484b77-48d2-4522-a074-1d814552fb11" containerName="payload-processing"
Oct 30 06:18:33.642177 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.642155    5066 cpu_manager.go:414] "containerMap: removing container" podUID="44d4c399-5239-4ac4-b4f8-a15ecf2313fc" containerName="maas-api"
Oct 30 06:18:33.642177 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.642160    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44d4c399-5239-4ac4-b4f8-a15ecf2313fc" containerName="maas-api"
Oct 30 06:18:33.642286 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.642181    5066 cpu_manager.go:414] "containerMap: removing container" podUID="b360e5eb-1915-47c0-9f8f-7adabbb6f52f" containerName="payload-pre-processing"
Oct 30 06:18:33.642286 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.642187    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b360e5eb-1915-47c0-9f8f-7adabbb6f52f" containerName="payload-pre-processing"
Oct 30 06:18:33.642351 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.642326    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="44d4c399-5239-4ac4-b4f8-a15ecf2313fc" containerName="maas-api"
Oct 30 06:18:33.642351 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.642340    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="b360e5eb-1915-47c0-9f8f-7adabbb6f52f" containerName="payload-pre-processing"
Oct 30 06:18:33.642414 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.642352    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="16484b77-48d2-4522-a074-1d814552fb11" containerName="payload-processing"
Oct 30 06:18:33.646014 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.645992    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:33.668286 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.668206    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"]
Oct 30 06:18:33.677076 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.677044    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-d51cd8-serving-cert\""
Oct 30 06:18:33.740260 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.740224    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:33.740449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.740347    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-metrics-certs\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:33.740449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.740371    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:33.740724 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.740700    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwgrh\" (UniqueName: \"kubernetes.io/projected/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-kube-api-access-hwgrh\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:33.794738 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.794700    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:33.858694 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.858653    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwgrh\" (UniqueName: \"kubernetes.io/projected/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-kube-api-access-hwgrh\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:33.862613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.859097    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:33.862972 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.862945    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-metrics-certs\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:33.863154 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.863098    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:33.875617 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:18:33.863451    5066 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-serving-cert: secret "maas-api-e2e-ipp-b-d51cd8-serving-cert" not found
Oct 30 06:18:33.875617 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:18:33.873862    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-maas-api-tls podName:ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9 nodeName:}" failed. No retries permitted until 2026-10-30 06:18:34.373830768 +0000 UTC m=+2818.119116104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-maas-api-tls") pod "maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76" (UID: "ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9") : secret "maas-api-e2e-ipp-b-d51cd8-serving-cert" not found
Oct 30 06:18:33.875617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.862778    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:33.884073 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.884032    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-metrics-certs\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:33.896001 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.895922    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwgrh\" (UniqueName: \"kubernetes.io/projected/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-kube-api-access-hwgrh\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:33.962038 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:33.961997    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:34.136042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:34.136010    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:34.473644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:34.473580    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:34.490440 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:34.490408    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:34.563654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:34.563084    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:34.881423 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:34.881391    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"]
Oct 30 06:18:34.882658 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:18:34.882628    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded3cb7a5_cf23_4257_89b4_5b693d2b2ef9.slice/crio-fa49cd48098f8900753cfb9d5ccb2e5fa3a1b64ed488140f6fa9f88dcf89eb8d WatchSource:0}: Error finding container fa49cd48098f8900753cfb9d5ccb2e5fa3a1b64ed488140f6fa9f88dcf89eb8d: Status 404 returned error can't find the container with id fa49cd48098f8900753cfb9d5ccb2e5fa3a1b64ed488140f6fa9f88dcf89eb8d
Oct 30 06:18:34.884216 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:34.884194    5066 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 06:18:34.946524 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:34.946487    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76" event={"ID":"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9","Type":"ContainerStarted","Data":"fa49cd48098f8900753cfb9d5ccb2e5fa3a1b64ed488140f6fa9f88dcf89eb8d"}
Oct 30 06:18:35.655405 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:35.654798    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:35.955550 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:35.955456    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76" event={"ID":"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9","Type":"ContainerStarted","Data":"f0556957b969b6992749e1ed5b9f8a037ca5254354733282c5f03d1309e47374"}
Oct 30 06:18:35.955966 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:35.955575    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:36.037435 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:36.037377    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76" podStartSLOduration=2.657065063 podStartE2EDuration="3.03735924s" totalImagesPullingTime="380.294177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:18:36.035242253 +0000 UTC m=+2819.780527585" watchObservedRunningTime="2026-10-30 06:18:36.03735924 +0000 UTC m=+2819.782644574"
Oct 30 06:18:37.649664 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:37.646949    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:39.414981 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:39.414949    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:41.200352 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:41.200272    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:41.969068 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:41.969039    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:18:42.592647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:42.592616    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"]
Oct 30 06:18:42.593165 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:42.592890    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk" podUID="abf5b5a4-32f5-4b5d-b524-632460da3bc1" containerName="maas-api" containerID="cri-o://b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc" gracePeriod=30
Oct 30 06:18:42.711944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:42.711916    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:42.902242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:42.902121    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:42.949824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:42.949802    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:18:43.006775 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.006741    5066 generic.go:356] "Generic (PLEG): container finished" podID="abf5b5a4-32f5-4b5d-b524-632460da3bc1" containerID="b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc" exitCode=0
Oct 30 06:18:43.006775 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.006787    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk" event={"ID":"abf5b5a4-32f5-4b5d-b524-632460da3bc1","Type":"ContainerDied","Data":"b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc"}
Oct 30 06:18:43.007036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.006787    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"
Oct 30 06:18:43.007036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.006838    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk" event={"ID":"abf5b5a4-32f5-4b5d-b524-632460da3bc1","Type":"ContainerDied","Data":"ecfb014591568de22c31c47769c6b11d5fdc3176e8fef960db9ea811c6fdab65"}
Oct 30 06:18:43.007036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.006870    5066 scope.go:122] "RemoveContainer" containerID="b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc"
Oct 30 06:18:43.020869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.020846    5066 scope.go:122] "RemoveContainer" containerID="b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc"
Oct 30 06:18:43.021284 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:18:43.021255    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc\": container with ID starting with b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc not found: ID does not exist" containerID="b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc"
Oct 30 06:18:43.021378 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.021294    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc"} err="failed to get container status \"b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc\": rpc error: code = NotFound desc = could not find container \"b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc\": container with ID starting with b3f91fde1e62016916e50429ae509601c336535451db666b6a5b68b52db824bc not found: ID does not exist"
Oct 30 06:18:43.034902 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.034868    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-metrics-certs\") pod \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") "
Oct 30 06:18:43.035043 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.034920    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/abf5b5a4-32f5-4b5d-b524-632460da3bc1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf5b5a4-32f5-4b5d-b524-632460da3bc1-openshift-service-ca\") pod \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") "
Oct 30 06:18:43.035158 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.035140    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abf5b5a4-32f5-4b5d-b524-632460da3bc1-kube-api-access-rl56d\" (UniqueName: \"kubernetes.io/projected/abf5b5a4-32f5-4b5d-b524-632460da3bc1-kube-api-access-rl56d\") pod \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") "
Oct 30 06:18:43.035243 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.035227    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-maas-api-tls\") pod \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\" (UID: \"abf5b5a4-32f5-4b5d-b524-632460da3bc1\") "
Oct 30 06:18:43.036446 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.036420    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abf5b5a4-32f5-4b5d-b524-632460da3bc1-openshift-service-ca" pod "abf5b5a4-32f5-4b5d-b524-632460da3bc1" (UID: "abf5b5a4-32f5-4b5d-b524-632460da3bc1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:18:43.040987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.040932    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf5b5a4-32f5-4b5d-b524-632460da3bc1-kube-api-access-rl56d" pod "abf5b5a4-32f5-4b5d-b524-632460da3bc1" (UID: "abf5b5a4-32f5-4b5d-b524-632460da3bc1"). InnerVolumeSpecName "kube-api-access-rl56d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:18:43.043102 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.043070    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-metrics-certs" pod "abf5b5a4-32f5-4b5d-b524-632460da3bc1" (UID: "abf5b5a4-32f5-4b5d-b524-632460da3bc1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:18:43.044526 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.044500    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-maas-api-tls" pod "abf5b5a4-32f5-4b5d-b524-632460da3bc1" (UID: "abf5b5a4-32f5-4b5d-b524-632460da3bc1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:18:43.092351 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.092313    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:43.142376 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.138042    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:43.142376 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.138081    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abf5b5a4-32f5-4b5d-b524-632460da3bc1-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:43.142376 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.138095    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abf5b5a4-32f5-4b5d-b524-632460da3bc1-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:43.142376 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.138110    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rl56d\" (UniqueName: \"kubernetes.io/projected/abf5b5a4-32f5-4b5d-b524-632460da3bc1-kube-api-access-rl56d\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:18:43.399559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.399526    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"]
Oct 30 06:18:43.400241 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.400225    5066 cpu_manager.go:414] "containerMap: removing container" podUID="abf5b5a4-32f5-4b5d-b524-632460da3bc1" containerName="maas-api"
Oct 30 06:18:43.400293 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.400243    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abf5b5a4-32f5-4b5d-b524-632460da3bc1" containerName="maas-api"
Oct 30 06:18:43.400366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.400357    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="abf5b5a4-32f5-4b5d-b524-632460da3bc1" containerName="maas-api"
Oct 30 06:18:43.405913 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.405880    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:43.424818 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.424789    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-5deb36-serving-cert\""
Oct 30 06:18:43.428166 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.428131    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:43.434370 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.434340    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"]
Oct 30 06:18:43.441401 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.441365    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"]
Oct 30 06:18:43.443180 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.443158    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-metrics-certs\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:43.443399 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.443373    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/191c6961-a046-467d-b8df-613be7c5dcb6-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:43.443491 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.443460    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n46b\" (UniqueName: \"kubernetes.io/projected/191c6961-a046-467d-b8df-613be7c5dcb6-kube-api-access-6n46b\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:43.443701 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.443680    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:43.444770 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.444750    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2d4576-86565c8454-7cmnk"]
Oct 30 06:18:43.545709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.545673    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-metrics-certs\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:43.545896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.545864    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/191c6961-a046-467d-b8df-613be7c5dcb6-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:43.545970 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.545953    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n46b\" (UniqueName: \"kubernetes.io/projected/191c6961-a046-467d-b8df-613be7c5dcb6-kube-api-access-6n46b\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:43.546162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.546140    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:43.546334 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:18:43.546317    5066 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-serving-cert: secret "maas-api-e2e-ipp-b-5deb36-serving-cert" not found
Oct 30 06:18:43.546414 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:18:43.546393    5066 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-maas-api-tls podName:191c6961-a046-467d-b8df-613be7c5dcb6 nodeName:}" failed. No retries permitted until 2026-10-30 06:18:44.046371142 +0000 UTC m=+2827.791656466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-maas-api-tls") pod "maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk" (UID: "191c6961-a046-467d-b8df-613be7c5dcb6") : secret "maas-api-e2e-ipp-b-5deb36-serving-cert" not found
Oct 30 06:18:43.546643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.546622    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/191c6961-a046-467d-b8df-613be7c5dcb6-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:43.549093 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.549046    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-metrics-certs\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:43.576785 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.576753    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n46b\" (UniqueName: \"kubernetes.io/projected/191c6961-a046-467d-b8df-613be7c5dcb6-kube-api-access-6n46b\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:43.649721 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.649634    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:43.832051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:43.832009    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:44.054010 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:44.053971    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:44.056801 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:44.056772    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-maas-api-tls\") pod \"maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:44.316727 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:44.316557    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:44.322694 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:44.322667    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:44.413789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:44.413753    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abf5b5a4-32f5-4b5d-b524-632460da3bc1" path="/var/lib/kubelet/pods/abf5b5a4-32f5-4b5d-b524-632460da3bc1/volumes"
Oct 30 06:18:44.641479 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:18:44.641445    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191c6961_a046_467d_b8df_613be7c5dcb6.slice/crio-021a0dd5516b2323882948ccc8f726eb7e188ec9a7c3e5778925c16bdb3cfff4 WatchSource:0}: Error finding container 021a0dd5516b2323882948ccc8f726eb7e188ec9a7c3e5778925c16bdb3cfff4: Status 404 returned error can't find the container with id 021a0dd5516b2323882948ccc8f726eb7e188ec9a7c3e5778925c16bdb3cfff4
Oct 30 06:18:44.642642 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:44.642615    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"]
Oct 30 06:18:45.050963 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:45.050923    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk" event={"ID":"191c6961-a046-467d-b8df-613be7c5dcb6","Type":"ContainerStarted","Data":"021a0dd5516b2323882948ccc8f726eb7e188ec9a7c3e5778925c16bdb3cfff4"}
Oct 30 06:18:45.927583 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:45.927544    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:46.059211 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:46.059177    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk" event={"ID":"191c6961-a046-467d-b8df-613be7c5dcb6","Type":"ContainerStarted","Data":"c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d"}
Oct 30 06:18:46.059602 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:46.059290    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:46.108079 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:46.108017    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk" podStartSLOduration=2.756539942 podStartE2EDuration="3.107997257s" totalImagesPullingTime="351.457315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:18:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:18:46.105112846 +0000 UTC m=+2829.850398177" watchObservedRunningTime="2026-10-30 06:18:46.107997257 +0000 UTC m=+2829.853282591"
Oct 30 06:18:47.947773 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:47.936235    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:49.619028 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:49.618984    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:50.370629 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:50.360376    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:50.533312 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:50.533277    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:50.711744 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:50.711711    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:51.465839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:51.464213    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:52.096618 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:52.095788    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:18:53.707747 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:53.707715    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:55.646161 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:55.646092    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:57.815141 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:57.815097    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:59.098352 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:59.098318    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:18:59.270763 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:18:59.267873    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:00.258922 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:00.258887    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:01.856996 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:01.856965    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:02.020993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:02.020957    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:02.186800 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:02.186740    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:02.554959 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:19:02.554878    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:19:04.045175 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:04.040195    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:05.759993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:05.759943    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:07.514486 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:07.514454    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:09.364084 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:09.364038    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:11.694571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:11.694527    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:13.439614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:13.438186    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:15.223661 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:15.219368    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:16.979495 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:16.979461    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:18.960124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:18.959299    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:21.532107 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:21.532062    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:23.344545 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:23.344509    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:25.127452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:25.127409    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:27.210632 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:27.208129    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:29.688651 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:29.685668    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:31.616650 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:31.616613    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:32.553884 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:19:32.553846    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:19:33.003810 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.003775    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"]
Oct 30 06:19:33.004328 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.004148    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb" podUID="2de87069-d3fc-46d6-965f-190c84575e91" containerName="maas-api" containerID="cri-o://4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a" gracePeriod=30
Oct 30 06:19:33.149999 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.149962    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:33.366772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.354705    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:33.424562 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.424224    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:19:33.486324 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.486281    5066 generic.go:356] "Generic (PLEG): container finished" podID="2de87069-d3fc-46d6-965f-190c84575e91" containerID="4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a" exitCode=0
Oct 30 06:19:33.486526 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.486357    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"
Oct 30 06:19:33.486618 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.486531    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb" event={"ID":"2de87069-d3fc-46d6-965f-190c84575e91","Type":"ContainerDied","Data":"4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a"}
Oct 30 06:19:33.486618 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.486581    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb" event={"ID":"2de87069-d3fc-46d6-965f-190c84575e91","Type":"ContainerDied","Data":"c78e223511e7f2bc5d7765dd6d2bb6b625c49056fe23c26e132d6f950e28d4d1"}
Oct 30 06:19:33.486756 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.486632    5066 scope.go:122] "RemoveContainer" containerID="4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a"
Oct 30 06:19:33.508112 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.508085    5066 scope.go:122] "RemoveContainer" containerID="4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a"
Oct 30 06:19:33.509102 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:19:33.509075    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a\": container with ID starting with 4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a not found: ID does not exist" containerID="4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a"
Oct 30 06:19:33.509314 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.509287    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a"} err="failed to get container status \"4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a\": rpc error: code = NotFound desc = could not find container \"4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a\": container with ID starting with 4441e0d6ec27c74b73206cad078d2751b8418a391a02f44dfc73680c2340b02a not found: ID does not exist"
Oct 30 06:19:33.533427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.533331    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2de87069-d3fc-46d6-965f-190c84575e91-kube-api-access-vnvst\" (UniqueName: \"kubernetes.io/projected/2de87069-d3fc-46d6-965f-190c84575e91-kube-api-access-vnvst\") pod \"2de87069-d3fc-46d6-965f-190c84575e91\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") "
Oct 30 06:19:33.533427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.533421    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-metrics-certs\") pod \"2de87069-d3fc-46d6-965f-190c84575e91\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") "
Oct 30 06:19:33.533686 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.533522    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-maas-api-tls\") pod \"2de87069-d3fc-46d6-965f-190c84575e91\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") "
Oct 30 06:19:33.533780 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.533756    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2de87069-d3fc-46d6-965f-190c84575e91-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de87069-d3fc-46d6-965f-190c84575e91-openshift-service-ca\") pod \"2de87069-d3fc-46d6-965f-190c84575e91\" (UID: \"2de87069-d3fc-46d6-965f-190c84575e91\") "
Oct 30 06:19:33.534330 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.534304    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de87069-d3fc-46d6-965f-190c84575e91-openshift-service-ca" pod "2de87069-d3fc-46d6-965f-190c84575e91" (UID: "2de87069-d3fc-46d6-965f-190c84575e91"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:19:33.534467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.534448    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2de87069-d3fc-46d6-965f-190c84575e91-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:33.537383 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.537343    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-maas-api-tls" pod "2de87069-d3fc-46d6-965f-190c84575e91" (UID: "2de87069-d3fc-46d6-965f-190c84575e91"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:19:33.537993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.537948    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-metrics-certs" pod "2de87069-d3fc-46d6-965f-190c84575e91" (UID: "2de87069-d3fc-46d6-965f-190c84575e91"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:19:33.539380 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.539355    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2de87069-d3fc-46d6-965f-190c84575e91-kube-api-access-vnvst" pod "2de87069-d3fc-46d6-965f-190c84575e91" (UID: "2de87069-d3fc-46d6-965f-190c84575e91"). InnerVolumeSpecName "kube-api-access-vnvst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:19:33.647612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.635875    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnvst\" (UniqueName: \"kubernetes.io/projected/2de87069-d3fc-46d6-965f-190c84575e91-kube-api-access-vnvst\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:33.647612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.635917    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:33.647612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.635932    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2de87069-d3fc-46d6-965f-190c84575e91-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:33.854121 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.854088    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"]
Oct 30 06:19:33.857545 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.857513    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-cfe14f-545bdbf888-dvvfb"]
Oct 30 06:19:33.980907 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:33.980863    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:34.159099 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:34.157794    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:34.405466 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:34.405433    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2de87069-d3fc-46d6-965f-190c84575e91" path="/var/lib/kubelet/pods/2de87069-d3fc-46d6-965f-190c84575e91/volumes"
Oct 30 06:19:34.674156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:34.674124    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:35.147936 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:35.147851    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:35.311740 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:35.311694    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:35.477795 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:35.477765    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:36.151281 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:36.145832    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:37.854148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:37.854098    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:39.636886 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:39.636852    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:40.999741 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:40.999698    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:41.161644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:41.161584    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:41.279311 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:41.279220    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"]
Oct 30 06:19:41.279799 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:41.279767    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr" podUID="173f259b-bfdc-465f-a4f5-b33e866932e2" containerName="istio-proxy" containerID="cri-o://83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a" gracePeriod=30
Oct 30 06:19:41.356270 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:41.356234    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:41.725532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:41.725480    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:42.017971 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:42.017863    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:42.190678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:42.190636    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:42.355403 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:42.355304    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:43.946443 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:43.946407    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:45.490071 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.490031    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"]
Oct 30 06:19:45.490978 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.490956    5066 cpu_manager.go:414] "containerMap: removing container" podUID="2de87069-d3fc-46d6-965f-190c84575e91" containerName="maas-api"
Oct 30 06:19:45.490978 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.490978    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2de87069-d3fc-46d6-965f-190c84575e91" containerName="maas-api"
Oct 30 06:19:45.491173 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.491146    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="2de87069-d3fc-46d6-965f-190c84575e91" containerName="maas-api"
Oct 30 06:19:45.496337 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.496315    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.520762 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.520732    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"]
Oct 30 06:19:45.589231 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.589199    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-certs\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.589403 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.589261    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.589496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.589475    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-token\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.589531 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.589513    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-envoy\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.589567 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.589539    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-podinfo\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.589703 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.589679    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istiod-ca-cert\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.589757 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.589744    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-data\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.589836 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.589816    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-credential-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.589894 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.589880    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmwbk\" (UniqueName: \"kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-kube-api-access-kmwbk\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.603083 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.603062    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-5deb36-openshift-default-dockercfg-7vlwt\""
Oct 30 06:19:45.691316 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.691284    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmwbk\" (UniqueName: \"kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-kube-api-access-kmwbk\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.691316 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.691320    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-certs\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.691546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.691359    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.691546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.691461    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-token\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.691546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.691481    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-envoy\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.691546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.691512    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-podinfo\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.691797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.691626    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istiod-ca-cert\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.691797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.691699    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-data\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.691797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.691746    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-credential-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.691958 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.691845    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-certs\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.691958 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.691898    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.692044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.692020    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-data\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.692116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.692091    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-credential-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.692543 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.692518    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istiod-ca-cert\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.693979 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.693959    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-envoy\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.694235 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.694217    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-podinfo\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.718086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.718058    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-token\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.718298 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.718274    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmwbk\" (UniqueName: \"kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-kube-api-access-kmwbk\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:45.784800 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.778489    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:45.822694 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:45.822625    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:46.148859 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:19:46.148829    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1d2da0_194d_44b7_94ad_c49bd3cb2451.slice/crio-3a665f7e9da7734d10e082438f015977c479579f82f59732b9a8a0088775951b WatchSource:0}: Error finding container 3a665f7e9da7734d10e082438f015977c479579f82f59732b9a8a0088775951b: Status 404 returned error can't find the container with id 3a665f7e9da7734d10e082438f015977c479579f82f59732b9a8a0088775951b
Oct 30 06:19:46.148994 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.148957    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"]
Oct 30 06:19:46.151749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.151719    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:19:46.151830 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.151786    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:19:46.151830 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.151815    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:19:46.578761 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.578732    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:19:46.624002 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.623965    5066 generic.go:356] "Generic (PLEG): container finished" podID="173f259b-bfdc-465f-a4f5-b33e866932e2" containerID="83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a" exitCode=0
Oct 30 06:19:46.627397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.624324    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"
Oct 30 06:19:46.627925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.624361    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr" event={"ID":"173f259b-bfdc-465f-a4f5-b33e866932e2","Type":"ContainerDied","Data":"83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a"}
Oct 30 06:19:46.628245 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.628219    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr" event={"ID":"173f259b-bfdc-465f-a4f5-b33e866932e2","Type":"ContainerDied","Data":"2e55f47e28aba482946e858adf3ab8b83af49c5e96afe3967646dc42fc68eb49"}
Oct 30 06:19:46.628531 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.628491    5066 scope.go:122] "RemoveContainer" containerID="83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a"
Oct 30 06:19:46.641831 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.641773    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc" event={"ID":"8a1d2da0-194d-44b7-94ad-c49bd3cb2451","Type":"ContainerStarted","Data":"5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5"}
Oct 30 06:19:46.642234 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.642174    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc" event={"ID":"8a1d2da0-194d-44b7-94ad-c49bd3cb2451","Type":"ContainerStarted","Data":"3a665f7e9da7734d10e082438f015977c479579f82f59732b9a8a0088775951b"}
Oct 30 06:19:46.652499 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.652475    5066 scope.go:122] "RemoveContainer" containerID="83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a"
Oct 30 06:19:46.652914 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:19:46.652864    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a\": container with ID starting with 83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a not found: ID does not exist" containerID="83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a"
Oct 30 06:19:46.652914 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.652900    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a"} err="failed to get container status \"83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a\": rpc error: code = NotFound desc = could not find container \"83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a\": container with ID starting with 83798451616d4af36c541c1cb541d79e02856fd276b7b0bbdc2e29b507575e3a not found: ID does not exist"
Oct 30 06:19:46.699079 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.698984    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc" podStartSLOduration=1.69897059 podStartE2EDuration="1.69897059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:19:46.698425611 +0000 UTC m=+2890.443710947" watchObservedRunningTime="2026-10-30 06:19:46.69897059 +0000 UTC m=+2890.444255973"
Oct 30 06:19:46.713128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.713084    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-socket\") pod \"173f259b-bfdc-465f-a4f5-b33e866932e2\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") "
Oct 30 06:19:46.713287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.713138    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-token\" (UniqueName: \"kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-token\") pod \"173f259b-bfdc-465f-a4f5-b33e866932e2\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") "
Oct 30 06:19:46.713287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.713198    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-envoy\") pod \"173f259b-bfdc-465f-a4f5-b33e866932e2\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") "
Oct 30 06:19:46.713287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.713232    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-credential-socket\") pod \"173f259b-bfdc-465f-a4f5-b33e866932e2\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") "
Oct 30 06:19:46.713458 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.713304    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/173f259b-bfdc-465f-a4f5-b33e866932e2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/173f259b-bfdc-465f-a4f5-b33e866932e2-istiod-ca-cert\") pod \"173f259b-bfdc-465f-a4f5-b33e866932e2\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") "
Oct 30 06:19:46.713458 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.713416    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-data\") pod \"173f259b-bfdc-465f-a4f5-b33e866932e2\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") "
Oct 30 06:19:46.713458 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.713442    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-certs\") pod \"173f259b-bfdc-465f-a4f5-b33e866932e2\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") "
Oct 30 06:19:46.713641 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.713480    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-kube-api-access-wcr2t\" (UniqueName: \"kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-kube-api-access-wcr2t\") pod \"173f259b-bfdc-465f-a4f5-b33e866932e2\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") "
Oct 30 06:19:46.713641 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.713520    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-podinfo\") pod \"173f259b-bfdc-465f-a4f5-b33e866932e2\" (UID: \"173f259b-bfdc-465f-a4f5-b33e866932e2\") "
Oct 30 06:19:46.714847 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.714095    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-certs" pod "173f259b-bfdc-465f-a4f5-b33e866932e2" (UID: "173f259b-bfdc-465f-a4f5-b33e866932e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:19:46.714847 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.714318    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-data" pod "173f259b-bfdc-465f-a4f5-b33e866932e2" (UID: "173f259b-bfdc-465f-a4f5-b33e866932e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:19:46.714847 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.714506    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:46.714847 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.714527    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:46.714847 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.714797    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-credential-socket" pod "173f259b-bfdc-465f-a4f5-b33e866932e2" (UID: "173f259b-bfdc-465f-a4f5-b33e866932e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:19:46.716126 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.716098    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/173f259b-bfdc-465f-a4f5-b33e866932e2-istiod-ca-cert" pod "173f259b-bfdc-465f-a4f5-b33e866932e2" (UID: "173f259b-bfdc-465f-a4f5-b33e866932e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:19:46.716496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.716473    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-socket" pod "173f259b-bfdc-465f-a4f5-b33e866932e2" (UID: "173f259b-bfdc-465f-a4f5-b33e866932e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:19:46.716728 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.716702    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-kube-api-access-wcr2t" pod "173f259b-bfdc-465f-a4f5-b33e866932e2" (UID: "173f259b-bfdc-465f-a4f5-b33e866932e2"). InnerVolumeSpecName "kube-api-access-wcr2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:19:46.716968 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.716939    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-podinfo" pod "173f259b-bfdc-465f-a4f5-b33e866932e2" (UID: "173f259b-bfdc-465f-a4f5-b33e866932e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:19:46.716968 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.716945    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-token" pod "173f259b-bfdc-465f-a4f5-b33e866932e2" (UID: "173f259b-bfdc-465f-a4f5-b33e866932e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:19:46.719235 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.719206    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-envoy" pod "173f259b-bfdc-465f-a4f5-b33e866932e2" (UID: "173f259b-bfdc-465f-a4f5-b33e866932e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:19:46.815662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.815624    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcr2t\" (UniqueName: \"kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-kube-api-access-wcr2t\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:46.815662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.815661    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:46.815899 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.815678    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:46.815899 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.815692    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:46.815899 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.815705    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:46.815899 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.815716    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/173f259b-bfdc-465f-a4f5-b33e866932e2-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:46.815899 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.815729    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/173f259b-bfdc-465f-a4f5-b33e866932e2-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:19:46.823293 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.823270    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:46.983208 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.983134    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"]
Oct 30 06:19:46.997328 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:46.997297    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-cfe14f-openshift-default-746f4c9457-vqkgr"]
Oct 30 06:19:47.704282 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:47.704224    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:47.839614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:47.839069    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.174:15021/healthz/ready\": dial tcp 10.132.0.174:15021: connect: connection refused" start-of-body=
Oct 30 06:19:47.839614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:47.839151    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc" podUID="8a1d2da0-194d-44b7-94ad-c49bd3cb2451" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.174:15021/healthz/ready\": dial tcp 10.132.0.174:15021: connect: connection refused"
Oct 30 06:19:48.013663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.011988    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:48.404242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.404219    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="173f259b-bfdc-465f-a4f5-b33e866932e2" path="/var/lib/kubelet/pods/173f259b-bfdc-465f-a4f5-b33e866932e2/volumes"
Oct 30 06:19:48.558922 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.556766    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"]
Oct 30 06:19:48.558922 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.557776    5066 cpu_manager.go:414] "containerMap: removing container" podUID="173f259b-bfdc-465f-a4f5-b33e866932e2" containerName="istio-proxy"
Oct 30 06:19:48.558922 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.557796    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="173f259b-bfdc-465f-a4f5-b33e866932e2" containerName="istio-proxy"
Oct 30 06:19:48.558922 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.558111    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="173f259b-bfdc-465f-a4f5-b33e866932e2" containerName="istio-proxy"
Oct 30 06:19:48.568292 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.567887    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"]
Oct 30 06:19:48.568292 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.568083    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.585883 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.585858    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-istio-token\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.586052 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.585925    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvlzk\" (UniqueName: \"kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-kube-api-access-bvlzk\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.586052 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.586019    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-envoy\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.586177 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.586140    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-certs\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.586256 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.586224    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f66efc60-7d24-4463-8668-6af9c0951b77-istio-podinfo\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.586312 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.586268    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f66efc60-7d24-4463-8668-6af9c0951b77-istiod-ca-cert\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.586312 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.586298    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-credential-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.586554 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.586530    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-data\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.586735 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.586581    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.655153 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.655037    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-d51cd8-openshift-default-dockercfg-gdp6w\""
Oct 30 06:19:48.694273 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.694223    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvlzk\" (UniqueName: \"kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-kube-api-access-bvlzk\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.694449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.694310    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-envoy\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.694449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.694343    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-certs\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.694449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.694415    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f66efc60-7d24-4463-8668-6af9c0951b77-istio-podinfo\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.694731 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.694453    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f66efc60-7d24-4463-8668-6af9c0951b77-istiod-ca-cert\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.694731 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.694482    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-credential-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.694850 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.694798    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-data\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.694911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.694866    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.694911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.694899    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-istio-token\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.696424 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.696295    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-certs\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.697658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.697572    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.698901 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.698570    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f66efc60-7d24-4463-8668-6af9c0951b77-istiod-ca-cert\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.698901 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.698864    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-credential-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.699556 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.699461    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-data\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.699770 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.699742    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f66efc60-7d24-4463-8668-6af9c0951b77-istio-podinfo\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.725636 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.724330    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvlzk\" (UniqueName: \"kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-kube-api-access-bvlzk\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.735985 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.735949    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-envoy\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.745390 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.745327    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-istio-token\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:48.872617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.828757    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:48.872617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.849723    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.174:15021/healthz/ready\": dial tcp 10.132.0.174:15021: connect: connection refused" start-of-body=
Oct 30 06:19:48.872617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.849784    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc" podUID="8a1d2da0-194d-44b7-94ad-c49bd3cb2451" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.174:15021/healthz/ready\": dial tcp 10.132.0.174:15021: connect: connection refused"
Oct 30 06:19:48.895362 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:48.895093    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:49.465256 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:49.465224    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"]
Oct 30 06:19:49.465924 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:19:49.465895    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66efc60_7d24_4463_8668_6af9c0951b77.slice/crio-c1230cd18feda48d3a40b206089c848f5afb750cb3202cd55aafd2e8527bc1cb WatchSource:0}: Error finding container c1230cd18feda48d3a40b206089c848f5afb750cb3202cd55aafd2e8527bc1cb: Status 404 returned error can't find the container with id c1230cd18feda48d3a40b206089c848f5afb750cb3202cd55aafd2e8527bc1cb
Oct 30 06:19:49.468914 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:49.468884    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:19:49.468991 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:49.468953    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:19:49.469027 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:49.469001    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:19:49.697701 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:49.695750    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb" event={"ID":"f66efc60-7d24-4463-8668-6af9c0951b77","Type":"ContainerStarted","Data":"c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58"}
Oct 30 06:19:49.697701 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:49.695816    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb" event={"ID":"f66efc60-7d24-4463-8668-6af9c0951b77","Type":"ContainerStarted","Data":"c1230cd18feda48d3a40b206089c848f5afb750cb3202cd55aafd2e8527bc1cb"}
Oct 30 06:19:49.759686 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:49.755764    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb" podStartSLOduration=1.755744375 podStartE2EDuration="1.755744375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:19:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:19:49.755498764 +0000 UTC m=+2893.500784098" watchObservedRunningTime="2026-10-30 06:19:49.755744375 +0000 UTC m=+2893.501029708"
Oct 30 06:19:49.827728 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:49.827692    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.174:15021/healthz/ready\": dial tcp 10.132.0.174:15021: connect: connection refused" start-of-body=
Oct 30 06:19:49.828000 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:49.827974    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc" podUID="8a1d2da0-194d-44b7-94ad-c49bd3cb2451" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.174:15021/healthz/ready\": dial tcp 10.132.0.174:15021: connect: connection refused"
Oct 30 06:19:49.861612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:49.859238    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:49.898993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:49.898962    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:50.823396 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:50.823362    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.174:15021/healthz/ready\": dial tcp 10.132.0.174:15021: connect: connection refused" start-of-body=
Oct 30 06:19:50.825965 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:50.823416    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc" podUID="8a1d2da0-194d-44b7-94ad-c49bd3cb2451" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.174:15021/healthz/ready\": dial tcp 10.132.0.174:15021: connect: connection refused"
Oct 30 06:19:50.897760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:50.897724    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.175:15021/healthz/ready\": dial tcp 10.132.0.175:15021: connect: connection refused" start-of-body=
Oct 30 06:19:50.897934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:50.897775    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb" podUID="f66efc60-7d24-4463-8668-6af9c0951b77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.175:15021/healthz/ready\": dial tcp 10.132.0.175:15021: connect: connection refused"
Oct 30 06:19:51.823538 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:51.823492    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.174:15021/healthz/ready\": dial tcp 10.132.0.174:15021: connect: connection refused" start-of-body=
Oct 30 06:19:51.844296 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:51.823550    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc" podUID="8a1d2da0-194d-44b7-94ad-c49bd3cb2451" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.174:15021/healthz/ready\": dial tcp 10.132.0.174:15021: connect: connection refused"
Oct 30 06:19:51.896116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:51.896079    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.175:15021/healthz/ready\": dial tcp 10.132.0.175:15021: connect: connection refused" start-of-body=
Oct 30 06:19:51.896300 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:51.896131    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb" podUID="f66efc60-7d24-4463-8668-6af9c0951b77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.175:15021/healthz/ready\": dial tcp 10.132.0.175:15021: connect: connection refused"
Oct 30 06:19:52.134674 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:52.134209    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:52.521477 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:52.521442    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"]
Oct 30 06:19:52.568235 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:52.568201    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"]
Oct 30 06:19:52.759915 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:52.759876    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:52.898074 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:52.897977    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.175:15021/healthz/ready\": dial tcp 10.132.0.175:15021: connect: connection refused" start-of-body=
Oct 30 06:19:52.898692 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:52.898658    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb" podUID="f66efc60-7d24-4463-8668-6af9c0951b77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.175:15021/healthz/ready\": dial tcp 10.132.0.175:15021: connect: connection refused"
Oct 30 06:19:52.954038 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:52.954006    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:52.955485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:52.955461    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:52.957816 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:52.957795    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:19:52.961165 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:52.961140    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:53.896353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:53.896314    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.175:15021/healthz/ready\": dial tcp 10.132.0.175:15021: connect: connection refused" start-of-body=
Oct 30 06:19:53.896532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:53.896370    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb" podUID="f66efc60-7d24-4463-8668-6af9c0951b77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.175:15021/healthz/ready\": dial tcp 10.132.0.175:15021: connect: connection refused"
Oct 30 06:19:54.896369 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:54.896338    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.175:15021/healthz/ready\": dial tcp 10.132.0.175:15021: connect: connection refused" start-of-body=
Oct 30 06:19:54.896790 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:54.896390    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb" podUID="f66efc60-7d24-4463-8668-6af9c0951b77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.175:15021/healthz/ready\": dial tcp 10.132.0.175:15021: connect: connection refused"
Oct 30 06:19:55.004517 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:55.004445    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:55.896640 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:55.896606    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.175:15021/healthz/ready\": dial tcp 10.132.0.175:15021: connect: connection refused" start-of-body=
Oct 30 06:19:55.897066 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:55.896658    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb" podUID="f66efc60-7d24-4463-8668-6af9c0951b77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.175:15021/healthz/ready\": dial tcp 10.132.0.175:15021: connect: connection refused"
Oct 30 06:19:56.850620 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:56.848234    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:56.901040 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:56.901009    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:56.901550 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:56.901416    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:56.902208 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:56.902187    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:19:57.522725 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:57.522689    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm" podUID="92cca1a6-0f7b-40db-bb56-dd8460b2706f" containerName="payload-processing" containerID="cri-o://ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a" gracePeriod=25
Oct 30 06:19:57.569706 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:57.569637    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt" podUID="8b5b79f3-9a52-443a-be06-ab7802f9263e" containerName="payload-pre-processing" containerID="cri-o://cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435" gracePeriod=25
Oct 30 06:19:58.756559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:58.756524    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:59.572866 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:59.572836    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:59.738287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:59.738249    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:19:59.988697 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:59.988661    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt" podUID="8b5b79f3-9a52-443a-be06-ab7802f9263e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 06:19:59.988697 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:19:59.988682    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm" podUID="92cca1a6-0f7b-40db-bb56-dd8460b2706f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 06:20:00.672416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:00.672376    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:02.477685 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:02.477648    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:02.558630 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:20:02.557579    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:20:04.438165 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:04.438068    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:06.745315 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:06.744997    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:08.506126 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:08.506085    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:09.988602 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:09.988557    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt" podUID="8b5b79f3-9a52-443a-be06-ab7802f9263e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 06:20:09.989065 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:09.988564    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm" podUID="92cca1a6-0f7b-40db-bb56-dd8460b2706f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 06:20:10.313386 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:10.313292    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:10.661918 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:10.661887    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"]
Oct 30 06:20:10.662186 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:10.662163    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" containerName="istio-proxy" containerID="cri-o://a1d93c8389727aaac91a9c6f6f0b51519318de5f913bc3239b7402a093d27468" gracePeriod=30
Oct 30 06:20:12.103493 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:12.103444    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:13.430139 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:13.430106    5066 patch_prober.go:29] interesting pod/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:20:13.430535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:13.430157    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:20:13.855896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:13.855793    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:15.916434 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:15.916400    5066 generic.go:356] "Generic (PLEG): container finished" podID="e09beacd-ed50-4e0d-b633-8900fae01778" containerID="a1d93c8389727aaac91a9c6f6f0b51519318de5f913bc3239b7402a093d27468" exitCode=0
Oct 30 06:20:15.916824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:15.916479    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" event={"ID":"e09beacd-ed50-4e0d-b633-8900fae01778","Type":"ContainerDied","Data":"a1d93c8389727aaac91a9c6f6f0b51519318de5f913bc3239b7402a093d27468"}
Oct 30 06:20:15.948161 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:15.948077    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 06:20:16.047282 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.047251    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e09beacd-ed50-4e0d-b633-8900fae01778-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e09beacd-ed50-4e0d-b633-8900fae01778-istiod-ca-cert\") pod \"e09beacd-ed50-4e0d-b633-8900fae01778\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") "
Oct 30 06:20:16.047491 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.047435    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-kube-api-access-klrh5\" (UniqueName: \"kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-kube-api-access-klrh5\") pod \"e09beacd-ed50-4e0d-b633-8900fae01778\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") "
Oct 30 06:20:16.047572 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.047513    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-envoy\") pod \"e09beacd-ed50-4e0d-b633-8900fae01778\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") "
Oct 30 06:20:16.047572 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.047550    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-credential-socket\") pod \"e09beacd-ed50-4e0d-b633-8900fae01778\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") "
Oct 30 06:20:16.047711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.047665    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-socket\") pod \"e09beacd-ed50-4e0d-b633-8900fae01778\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") "
Oct 30 06:20:16.047770 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.047706    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e09beacd-ed50-4e0d-b633-8900fae01778-istiod-ca-cert" pod "e09beacd-ed50-4e0d-b633-8900fae01778" (UID: "e09beacd-ed50-4e0d-b633-8900fae01778"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:20:16.047770 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.047723    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-istio-token\" (UniqueName: \"kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-istio-token\") pod \"e09beacd-ed50-4e0d-b633-8900fae01778\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") "
Oct 30 06:20:16.047892 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.047794    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e09beacd-ed50-4e0d-b633-8900fae01778-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e09beacd-ed50-4e0d-b633-8900fae01778-istio-podinfo\") pod \"e09beacd-ed50-4e0d-b633-8900fae01778\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") "
Oct 30 06:20:16.047892 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.047839    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-data\") pod \"e09beacd-ed50-4e0d-b633-8900fae01778\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") "
Oct 30 06:20:16.047892 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.047880    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-certs\") pod \"e09beacd-ed50-4e0d-b633-8900fae01778\" (UID: \"e09beacd-ed50-4e0d-b633-8900fae01778\") "
Oct 30 06:20:16.048049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.047988    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-credential-socket" pod "e09beacd-ed50-4e0d-b633-8900fae01778" (UID: "e09beacd-ed50-4e0d-b633-8900fae01778"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:20:16.048049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.048036    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-socket" pod "e09beacd-ed50-4e0d-b633-8900fae01778" (UID: "e09beacd-ed50-4e0d-b633-8900fae01778"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:20:16.048665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.048643    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:16.048840 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.048825    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e09beacd-ed50-4e0d-b633-8900fae01778-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:16.048947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.048935    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:16.049330 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.049310    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-certs" pod "e09beacd-ed50-4e0d-b633-8900fae01778" (UID: "e09beacd-ed50-4e0d-b633-8900fae01778"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:20:16.049466 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.049332    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-data" pod "e09beacd-ed50-4e0d-b633-8900fae01778" (UID: "e09beacd-ed50-4e0d-b633-8900fae01778"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:20:16.050842 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.050816    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-envoy" pod "e09beacd-ed50-4e0d-b633-8900fae01778" (UID: "e09beacd-ed50-4e0d-b633-8900fae01778"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:20:16.054412 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.054384    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e09beacd-ed50-4e0d-b633-8900fae01778-istio-podinfo" pod "e09beacd-ed50-4e0d-b633-8900fae01778" (UID: "e09beacd-ed50-4e0d-b633-8900fae01778"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:20:16.054505 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.054431    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-kube-api-access-klrh5" pod "e09beacd-ed50-4e0d-b633-8900fae01778" (UID: "e09beacd-ed50-4e0d-b633-8900fae01778"). InnerVolumeSpecName "kube-api-access-klrh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:20:16.054505 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.054456    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-istio-token" pod "e09beacd-ed50-4e0d-b633-8900fae01778" (UID: "e09beacd-ed50-4e0d-b633-8900fae01778"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:20:16.150620 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.150555    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klrh5\" (UniqueName: \"kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-kube-api-access-klrh5\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:16.150620 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.150583    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:16.150833 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.150636    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e09beacd-ed50-4e0d-b633-8900fae01778-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:16.150833 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.150644    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e09beacd-ed50-4e0d-b633-8900fae01778-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:16.150833 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.150653    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:16.150833 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.150661    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e09beacd-ed50-4e0d-b633-8900fae01778-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:16.380766 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.380646    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:16.926850 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.926823    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"
Oct 30 06:20:16.926850 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.926844    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz" event={"ID":"e09beacd-ed50-4e0d-b633-8900fae01778","Type":"ContainerDied","Data":"9f18d6fe44a8c243485be7dadda02d90054b90492555f1f7301caa4f9f595c95"}
Oct 30 06:20:16.927416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:16.926905    5066 scope.go:122] "RemoveContainer" containerID="a1d93c8389727aaac91a9c6f6f0b51519318de5f913bc3239b7402a093d27468"
Oct 30 06:20:17.001094 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.001060    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"]
Oct 30 06:20:17.015336 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.015294    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-4a68a1-openshift-default-64ff97f89c-8mgrz"]
Oct 30 06:20:17.896217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.896191    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"
Oct 30 06:20:17.916120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.916096    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"
Oct 30 06:20:17.936125 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.936093    5066 generic.go:356] "Generic (PLEG): container finished" podID="8b5b79f3-9a52-443a-be06-ab7802f9263e" containerID="cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435" exitCode=0
Oct 30 06:20:17.936530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.936129    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"
Oct 30 06:20:17.936530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.936180    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt" event={"ID":"8b5b79f3-9a52-443a-be06-ab7802f9263e","Type":"ContainerDied","Data":"cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435"}
Oct 30 06:20:17.936530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.936231    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt" event={"ID":"8b5b79f3-9a52-443a-be06-ab7802f9263e","Type":"ContainerDied","Data":"d7b1bc83a5dbd58da134f58453986a597f8ecfb6ea4629adb5d6bde24e92669a"}
Oct 30 06:20:17.936530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.936258    5066 scope.go:122] "RemoveContainer" containerID="cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435"
Oct 30 06:20:17.937639 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.937619    5066 generic.go:356] "Generic (PLEG): container finished" podID="92cca1a6-0f7b-40db-bb56-dd8460b2706f" containerID="ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a" exitCode=0
Oct 30 06:20:17.937716 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.937658    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"
Oct 30 06:20:17.937716 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.937661    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm" event={"ID":"92cca1a6-0f7b-40db-bb56-dd8460b2706f","Type":"ContainerDied","Data":"ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a"}
Oct 30 06:20:17.937820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.937714    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm" event={"ID":"92cca1a6-0f7b-40db-bb56-dd8460b2706f","Type":"ContainerDied","Data":"42c5f0731602260d0d3faa13a3cd79f2851de3c91788d6f16fbdefa58a365f16"}
Oct 30 06:20:17.949038 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.949016    5066 scope.go:122] "RemoveContainer" containerID="cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435"
Oct 30 06:20:17.949395 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:20:17.949370    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435\": container with ID starting with cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435 not found: ID does not exist" containerID="cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435"
Oct 30 06:20:17.949463 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.949407    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435"} err="failed to get container status \"cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435\": rpc error: code = NotFound desc = could not find container \"cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435\": container with ID starting with cfb8af72be5867fec9c77e14f4ad9bf31a75ec7e5007dc784a6fd6baf9d91435 not found: ID does not exist"
Oct 30 06:20:17.949463 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.949430    5066 scope.go:122] "RemoveContainer" containerID="ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a"
Oct 30 06:20:17.969000 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.968910    5066 scope.go:122] "RemoveContainer" containerID="ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a"
Oct 30 06:20:17.969417 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:20:17.969388    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a\": container with ID starting with ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a not found: ID does not exist" containerID="ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a"
Oct 30 06:20:17.969545 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.969422    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a"} err="failed to get container status \"ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a\": rpc error: code = NotFound desc = could not find container \"ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a\": container with ID starting with ff61643c409472ee528345f458b627724282604230416e5b3a0da292b8f0e51a not found: ID does not exist"
Oct 30 06:20:17.990021 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.989796    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92cca1a6-0f7b-40db-bb56-dd8460b2706f-kube-api-access-jg4dx\" (UniqueName: \"kubernetes.io/projected/92cca1a6-0f7b-40db-bb56-dd8460b2706f-kube-api-access-jg4dx\") pod \"92cca1a6-0f7b-40db-bb56-dd8460b2706f\" (UID: \"92cca1a6-0f7b-40db-bb56-dd8460b2706f\") "
Oct 30 06:20:17.990021 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.989910    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b5b79f3-9a52-443a-be06-ab7802f9263e-kube-api-access-g5fqd\" (UniqueName: \"kubernetes.io/projected/8b5b79f3-9a52-443a-be06-ab7802f9263e-kube-api-access-g5fqd\") pod \"8b5b79f3-9a52-443a-be06-ab7802f9263e\" (UID: \"8b5b79f3-9a52-443a-be06-ab7802f9263e\") "
Oct 30 06:20:17.990370 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.990309    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92cca1a6-0f7b-40db-bb56-dd8460b2706f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92cca1a6-0f7b-40db-bb56-dd8460b2706f-plugins-config-volume\") pod \"92cca1a6-0f7b-40db-bb56-dd8460b2706f\" (UID: \"92cca1a6-0f7b-40db-bb56-dd8460b2706f\") "
Oct 30 06:20:17.990449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.990375    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b5b79f3-9a52-443a-be06-ab7802f9263e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b5b79f3-9a52-443a-be06-ab7802f9263e-plugins-config-volume\") pod \"8b5b79f3-9a52-443a-be06-ab7802f9263e\" (UID: \"8b5b79f3-9a52-443a-be06-ab7802f9263e\") "
Oct 30 06:20:17.991931 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.991819    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92cca1a6-0f7b-40db-bb56-dd8460b2706f-plugins-config-volume" pod "92cca1a6-0f7b-40db-bb56-dd8460b2706f" (UID: "92cca1a6-0f7b-40db-bb56-dd8460b2706f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:20:17.991931 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.991892    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b5b79f3-9a52-443a-be06-ab7802f9263e-plugins-config-volume" pod "8b5b79f3-9a52-443a-be06-ab7802f9263e" (UID: "8b5b79f3-9a52-443a-be06-ab7802f9263e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:20:17.992311 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.992288    5066 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92cca1a6-0f7b-40db-bb56-dd8460b2706f-plugins-config-volume\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:17.992627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.992316    5066 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b5b79f3-9a52-443a-be06-ab7802f9263e-plugins-config-volume\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:17.995070 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.995042    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92cca1a6-0f7b-40db-bb56-dd8460b2706f-kube-api-access-jg4dx" pod "92cca1a6-0f7b-40db-bb56-dd8460b2706f" (UID: "92cca1a6-0f7b-40db-bb56-dd8460b2706f"). InnerVolumeSpecName "kube-api-access-jg4dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:20:17.995505 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:17.995483    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b5b79f3-9a52-443a-be06-ab7802f9263e-kube-api-access-g5fqd" pod "8b5b79f3-9a52-443a-be06-ab7802f9263e" (UID: "8b5b79f3-9a52-443a-be06-ab7802f9263e"). InnerVolumeSpecName "kube-api-access-g5fqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:20:18.094261 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:18.094167    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg4dx\" (UniqueName: \"kubernetes.io/projected/92cca1a6-0f7b-40db-bb56-dd8460b2706f-kube-api-access-jg4dx\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:18.094261 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:18.094194    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5fqd\" (UniqueName: \"kubernetes.io/projected/8b5b79f3-9a52-443a-be06-ab7802f9263e-kube-api-access-g5fqd\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:18.282978 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:18.282935    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:18.337825 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:18.337795    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"]
Oct 30 06:20:18.341293 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:18.341269    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-03d349-6747675b67-qzzqt"]
Oct 30 06:20:18.378392 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:18.378314    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"]
Oct 30 06:20:18.387192 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:18.387159    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-03d349-77c695ffdf-dfjbm"]
Oct 30 06:20:18.402530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:18.402498    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b5b79f3-9a52-443a-be06-ab7802f9263e" path="/var/lib/kubelet/pods/8b5b79f3-9a52-443a-be06-ab7802f9263e/volumes"
Oct 30 06:20:18.402956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:18.402942    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92cca1a6-0f7b-40db-bb56-dd8460b2706f" path="/var/lib/kubelet/pods/92cca1a6-0f7b-40db-bb56-dd8460b2706f/volumes"
Oct 30 06:20:18.403371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:18.403359    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" path="/var/lib/kubelet/pods/e09beacd-ed50-4e0d-b633-8900fae01778/volumes"
Oct 30 06:20:19.972628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:19.971697    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:21.297662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.297631    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"]
Oct 30 06:20:21.298185 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.297844    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4" podUID="c634e3e8-f5df-4c8b-a929-ac74517bd9e8" containerName="maas-api" containerID="cri-o://d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be" gracePeriod=30
Oct 30 06:20:21.412314 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.412283    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:21.608912 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.608667    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:21.639625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.639558    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:20:21.746120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.746086    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-kube-api-access-w762d\" (UniqueName: \"kubernetes.io/projected/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-kube-api-access-w762d\") pod \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") "
Oct 30 06:20:21.746341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.746189    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-metrics-certs\") pod \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") "
Oct 30 06:20:21.746341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.746265    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-openshift-service-ca\") pod \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") "
Oct 30 06:20:21.746341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.746327    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-maas-api-tls\") pod \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\" (UID: \"c634e3e8-f5df-4c8b-a929-ac74517bd9e8\") "
Oct 30 06:20:21.747554 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.747521    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-openshift-service-ca" pod "c634e3e8-f5df-4c8b-a929-ac74517bd9e8" (UID: "c634e3e8-f5df-4c8b-a929-ac74517bd9e8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:20:21.749737 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.749706    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-maas-api-tls" pod "c634e3e8-f5df-4c8b-a929-ac74517bd9e8" (UID: "c634e3e8-f5df-4c8b-a929-ac74517bd9e8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:20:21.749993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.749966    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-metrics-certs" pod "c634e3e8-f5df-4c8b-a929-ac74517bd9e8" (UID: "c634e3e8-f5df-4c8b-a929-ac74517bd9e8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:20:21.750972 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.750951    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-kube-api-access-w762d" pod "c634e3e8-f5df-4c8b-a929-ac74517bd9e8" (UID: "c634e3e8-f5df-4c8b-a929-ac74517bd9e8"). InnerVolumeSpecName "kube-api-access-w762d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:20:21.772698 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.772664    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:21.852099 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.851937    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:21.852099 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.851984    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:21.852099 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.852000    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w762d\" (UniqueName: \"kubernetes.io/projected/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-kube-api-access-w762d\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:21.852099 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:21.852013    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c634e3e8-f5df-4c8b-a929-ac74517bd9e8-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:22.002745 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.002713    5066 generic.go:356] "Generic (PLEG): container finished" podID="c634e3e8-f5df-4c8b-a929-ac74517bd9e8" containerID="d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be" exitCode=0
Oct 30 06:20:22.003136 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.003122    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"
Oct 30 06:20:22.005576 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.005548    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4" event={"ID":"c634e3e8-f5df-4c8b-a929-ac74517bd9e8","Type":"ContainerDied","Data":"d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be"}
Oct 30 06:20:22.005813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.005788    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4" event={"ID":"c634e3e8-f5df-4c8b-a929-ac74517bd9e8","Type":"ContainerDied","Data":"2c2ecf0107ac60eb370c5e970f9f14a6ef8be820480c8791b94727563137b61e"}
Oct 30 06:20:22.005930 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.005854    5066 scope.go:122] "RemoveContainer" containerID="d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be"
Oct 30 06:20:22.020453 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.020434    5066 scope.go:122] "RemoveContainer" containerID="d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be"
Oct 30 06:20:22.020839 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:20:22.020812    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be\": container with ID starting with d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be not found: ID does not exist" containerID="d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be"
Oct 30 06:20:22.020911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.020850    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be"} err="failed to get container status \"d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be\": rpc error: code = NotFound desc = could not find container \"d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be\": container with ID starting with d64c94fd6d70db82d91cbb855c050aaa9b8d9fee596807e2d874883bd0cf51be not found: ID does not exist"
Oct 30 06:20:22.066210 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.066176    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"]
Oct 30 06:20:22.079098 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.079066    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:22.080497 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.080467    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-dddc53-5d886d89b4-jfhn4"]
Oct 30 06:20:22.250973 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.250931    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:22.404903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.404864    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c634e3e8-f5df-4c8b-a929-ac74517bd9e8" path="/var/lib/kubelet/pods/c634e3e8-f5df-4c8b-a929-ac74517bd9e8/volumes"
Oct 30 06:20:22.414282 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.414257    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:22.781241 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:22.781210    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:23.795074 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:23.795030    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:23.963210 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:23.963173    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:24.308421 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.308371    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"]
Oct 30 06:20:24.309613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.309562    5066 cpu_manager.go:414] "containerMap: removing container" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" containerName="istio-proxy"
Oct 30 06:20:24.309814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.309656    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" containerName="istio-proxy"
Oct 30 06:20:24.309814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.309714    5066 cpu_manager.go:414] "containerMap: removing container" podUID="8b5b79f3-9a52-443a-be06-ab7802f9263e" containerName="payload-pre-processing"
Oct 30 06:20:24.309814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.309723    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b5b79f3-9a52-443a-be06-ab7802f9263e" containerName="payload-pre-processing"
Oct 30 06:20:24.309814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.309749    5066 cpu_manager.go:414] "containerMap: removing container" podUID="92cca1a6-0f7b-40db-bb56-dd8460b2706f" containerName="payload-processing"
Oct 30 06:20:24.309814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.309758    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92cca1a6-0f7b-40db-bb56-dd8460b2706f" containerName="payload-processing"
Oct 30 06:20:24.309814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.309775    5066 cpu_manager.go:414] "containerMap: removing container" podUID="c634e3e8-f5df-4c8b-a929-ac74517bd9e8" containerName="maas-api"
Oct 30 06:20:24.309814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.309784    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c634e3e8-f5df-4c8b-a929-ac74517bd9e8" containerName="maas-api"
Oct 30 06:20:24.310482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.310084    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b5b79f3-9a52-443a-be06-ab7802f9263e" containerName="payload-pre-processing"
Oct 30 06:20:24.310482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.310108    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="92cca1a6-0f7b-40db-bb56-dd8460b2706f" containerName="payload-processing"
Oct 30 06:20:24.310482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.310125    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="c634e3e8-f5df-4c8b-a929-ac74517bd9e8" containerName="maas-api"
Oct 30 06:20:24.310482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.310149    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="e09beacd-ed50-4e0d-b633-8900fae01778" containerName="istio-proxy"
Oct 30 06:20:24.325416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.322074    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"
Oct 30 06:20:24.337470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.337438    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"]
Oct 30 06:20:24.395175 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.395138    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-d51cd8-dockercfg-kffjc\""
Oct 30 06:20:24.406288 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.406257    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-d51cd8\""
Oct 30 06:20:24.416148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.416118    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43685c17-86ec-4839-bb32-7251aba4e869-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99\" (UID: \"43685c17-86ec-4839-bb32-7251aba4e869\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"
Oct 30 06:20:24.416325 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.416307    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4sfz\" (UniqueName: \"kubernetes.io/projected/43685c17-86ec-4839-bb32-7251aba4e869-kube-api-access-n4sfz\") pod \"payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99\" (UID: \"43685c17-86ec-4839-bb32-7251aba4e869\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"
Oct 30 06:20:24.518370 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.518339    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4sfz\" (UniqueName: \"kubernetes.io/projected/43685c17-86ec-4839-bb32-7251aba4e869-kube-api-access-n4sfz\") pod \"payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99\" (UID: \"43685c17-86ec-4839-bb32-7251aba4e869\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"
Oct 30 06:20:24.518613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.518553    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43685c17-86ec-4839-bb32-7251aba4e869-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99\" (UID: \"43685c17-86ec-4839-bb32-7251aba4e869\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"
Oct 30 06:20:24.519431 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.519408    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43685c17-86ec-4839-bb32-7251aba4e869-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99\" (UID: \"43685c17-86ec-4839-bb32-7251aba4e869\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"
Oct 30 06:20:24.544032 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.544003    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4sfz\" (UniqueName: \"kubernetes.io/projected/43685c17-86ec-4839-bb32-7251aba4e869-kube-api-access-n4sfz\") pod \"payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99\" (UID: \"43685c17-86ec-4839-bb32-7251aba4e869\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"
Oct 30 06:20:24.644282 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.644187    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"
Oct 30 06:20:24.952241 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:24.952218    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"]
Oct 30 06:20:24.953420 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:20:24.953394    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43685c17_86ec_4839_bb32_7251aba4e869.slice/crio-39c725a96b1bcdbda1e7fd89c9fb65ad608badc7ae9387e2481b60a3d103d361 WatchSource:0}: Error finding container 39c725a96b1bcdbda1e7fd89c9fb65ad608badc7ae9387e2481b60a3d103d361: Status 404 returned error can't find the container with id 39c725a96b1bcdbda1e7fd89c9fb65ad608badc7ae9387e2481b60a3d103d361
Oct 30 06:20:25.029408 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:25.029266    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99" event={"ID":"43685c17-86ec-4839-bb32-7251aba4e869","Type":"ContainerStarted","Data":"39c725a96b1bcdbda1e7fd89c9fb65ad608badc7ae9387e2481b60a3d103d361"}
Oct 30 06:20:25.076754 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:25.076382    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:26.037823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:26.037788    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99" event={"ID":"43685c17-86ec-4839-bb32-7251aba4e869","Type":"ContainerStarted","Data":"32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c"}
Oct 30 06:20:26.038235 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:26.037912    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"
Oct 30 06:20:26.086692 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:26.086646    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99" podStartSLOduration=2.086631474 podStartE2EDuration="2.086631474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:20:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:20:26.083283573 +0000 UTC m=+2929.828568911" watchObservedRunningTime="2026-10-30 06:20:26.086631474 +0000 UTC m=+2929.831916835"
Oct 30 06:20:26.801613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:26.799315    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:28.762621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:28.762489    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:29.055346 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:29.055302    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:29.231662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:29.231629    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:29.402876 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:29.402777    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:29.601616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:29.596131    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:30.621226 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.621189    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"]
Oct 30 06:20:30.625920 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.625900    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.661344 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.661293    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.661344 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.661344    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-envoy\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.661626 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.661379    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-podinfo\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.661626 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.661476    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhsrx\" (UniqueName: \"kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-kube-api-access-xhsrx\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.661626 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.661538    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-data\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.661802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.661736    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-credential-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.661860 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.661804    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7295343-bb58-43a6-a460-9b26fb26edc0-istiod-ca-cert\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.661912 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.661890    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-certs\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.661972 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.661916    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-token\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.665799 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.665540    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"]
Oct 30 06:20:30.763099 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.763067    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhsrx\" (UniqueName: \"kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-kube-api-access-xhsrx\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.763284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.763118    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-data\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.763284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.763182    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-credential-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.763284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.763218    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7295343-bb58-43a6-a460-9b26fb26edc0-istiod-ca-cert\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.763284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.763266    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-certs\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.763284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.763281    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-token\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.763754 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.763388    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.763754 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.763415    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-envoy\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.763754 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.763443    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-podinfo\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.763898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.763855    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-credential-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.764449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.764238    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-certs\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.764449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.764418    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.764449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.763824    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-data\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.764727 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.764421    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7295343-bb58-43a6-a460-9b26fb26edc0-istiod-ca-cert\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.767123 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.767095    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-envoy\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.770082 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.770057    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-podinfo\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.815353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.810293    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-token\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.823632 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.823026    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhsrx\" (UniqueName: \"kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-kube-api-access-xhsrx\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:30.829380 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.828173    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:30.965150 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:30.965106    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:31.292567 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:31.292540    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"]
Oct 30 06:20:31.292814 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:20:31.292793    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7295343_bb58_43a6_a460_9b26fb26edc0.slice/crio-5e353f848851b4410949f7b7e5eb9b625f997cfcf5b032dfb5cc040d7cf12acc WatchSource:0}: Error finding container 5e353f848851b4410949f7b7e5eb9b625f997cfcf5b032dfb5cc040d7cf12acc: Status 404 returned error can't find the container with id 5e353f848851b4410949f7b7e5eb9b625f997cfcf5b032dfb5cc040d7cf12acc
Oct 30 06:20:31.295800 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:31.295765    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:20:31.295909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:31.295850    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:20:31.295909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:31.295896    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:20:32.107865 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.107822    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n" event={"ID":"c7295343-bb58-43a6-a460-9b26fb26edc0","Type":"ContainerStarted","Data":"44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d"}
Oct 30 06:20:32.108295 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.107889    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n" event={"ID":"c7295343-bb58-43a6-a460-9b26fb26edc0","Type":"ContainerStarted","Data":"5e353f848851b4410949f7b7e5eb9b625f997cfcf5b032dfb5cc040d7cf12acc"}
Oct 30 06:20:32.184630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.184562    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n" podStartSLOduration=2.184544268 podStartE2EDuration="2.184544268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:20:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:20:32.182829911 +0000 UTC m=+2935.928115243" watchObservedRunningTime="2026-10-30 06:20:32.184544268 +0000 UTC m=+2935.929829600"
Oct 30 06:20:32.349757 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.349705    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:32.547208 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.529652    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:32.598093 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:20:32.598061    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:20:32.873674 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.870740    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"]
Oct 30 06:20:32.905613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.890000    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"]
Oct 30 06:20:32.905613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.890143    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"
Oct 30 06:20:32.934616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.917881    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf77c510-85ab-4b5c-a7a3-a62d388ab023-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg\" (UID: \"bf77c510-85ab-4b5c-a7a3-a62d388ab023\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"
Oct 30 06:20:32.934616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.917975    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcm4t\" (UniqueName: \"kubernetes.io/projected/bf77c510-85ab-4b5c-a7a3-a62d388ab023-kube-api-access-xcm4t\") pod \"payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg\" (UID: \"bf77c510-85ab-4b5c-a7a3-a62d388ab023\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"
Oct 30 06:20:32.956613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.956356    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-5deb36-dockercfg-kqrfh\""
Oct 30 06:20:32.956613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.956455    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-5deb36\""
Oct 30 06:20:32.980630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.976824    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:32.985555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.985172    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 06:20:32.985555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:32.985232    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n" podUID="c7295343-bb58-43a6-a460-9b26fb26edc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 06:20:33.019967 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.019925    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf77c510-85ab-4b5c-a7a3-a62d388ab023-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg\" (UID: \"bf77c510-85ab-4b5c-a7a3-a62d388ab023\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"
Oct 30 06:20:33.020193 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.020034    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcm4t\" (UniqueName: \"kubernetes.io/projected/bf77c510-85ab-4b5c-a7a3-a62d388ab023-kube-api-access-xcm4t\") pod \"payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg\" (UID: \"bf77c510-85ab-4b5c-a7a3-a62d388ab023\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"
Oct 30 06:20:33.020775 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.020754    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf77c510-85ab-4b5c-a7a3-a62d388ab023-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg\" (UID: \"bf77c510-85ab-4b5c-a7a3-a62d388ab023\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"
Oct 30 06:20:33.072628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.067952    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcm4t\" (UniqueName: \"kubernetes.io/projected/bf77c510-85ab-4b5c-a7a3-a62d388ab023-kube-api-access-xcm4t\") pod \"payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg\" (UID: \"bf77c510-85ab-4b5c-a7a3-a62d388ab023\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"
Oct 30 06:20:33.301219 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.301185    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"
Oct 30 06:20:33.351257 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.351067    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:33.597705 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.597456    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:33.659646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.659435    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"]
Oct 30 06:20:33.701203 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.701173    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"]
Oct 30 06:20:33.707020 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.706993    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.762709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.761666    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxp5c\" (UniqueName: \"kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-kube-api-access-xxp5c\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.762709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.761732    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-data\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.762709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.761829    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.762709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.761922    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-istio-token\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.762709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.761988    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c98c4321-bd50-417b-ae05-8def39068949-istiod-ca-cert\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.762709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.762056    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-credential-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.762709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.762139    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c98c4321-bd50-417b-ae05-8def39068949-istio-podinfo\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.762709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.762180    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-envoy\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.762709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.762205    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-certs\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.770287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.770252    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"]
Oct 30 06:20:33.865416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.865316    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c98c4321-bd50-417b-ae05-8def39068949-istio-podinfo\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.866284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.866258    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-envoy\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.866903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.866884    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-certs\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.867508 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.867336    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-certs\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.867746 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.867725    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxp5c\" (UniqueName: \"kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-kube-api-access-xxp5c\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.868250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.868229    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-data\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.868826 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.868679    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-data\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.869079 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.869061    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.869615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.869454    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.869864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.869843    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-istio-token\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.870327 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.870307    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c98c4321-bd50-417b-ae05-8def39068949-istiod-ca-cert\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.871249 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.871116    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c98c4321-bd50-417b-ae05-8def39068949-istiod-ca-cert\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.871779 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.871725    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-credential-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.871912 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.871892    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-credential-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.873065 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.872379    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-envoy\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.873065 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.872838    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c98c4321-bd50-417b-ae05-8def39068949-istio-podinfo\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.893716 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.893687    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxp5c\" (UniqueName: \"kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-kube-api-access-xxp5c\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.894994 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.894976    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-istio-token\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:33.966263 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.966231    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 06:20:33.966432 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:33.966290    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n" podUID="c7295343-bb58-43a6-a460-9b26fb26edc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 06:20:34.070194 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:34.070165    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:34.160027 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:34.159881    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg" event={"ID":"bf77c510-85ab-4b5c-a7a3-a62d388ab023","Type":"ContainerStarted","Data":"bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009"}
Oct 30 06:20:34.160027 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:34.159933    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg" event={"ID":"bf77c510-85ab-4b5c-a7a3-a62d388ab023","Type":"ContainerStarted","Data":"89b08c34cec654d63ff886ad6df4e6e0d37a72a3fa303acff6a29a442046679e"}
Oct 30 06:20:34.160027 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:34.159979    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"
Oct 30 06:20:34.208607 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:34.208527    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg" podStartSLOduration=2.208507445 podStartE2EDuration="2.208507445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:20:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:20:34.207657774 +0000 UTC m=+2937.952943108" watchObservedRunningTime="2026-10-30 06:20:34.208507445 +0000 UTC m=+2937.953792775"
Oct 30 06:20:34.294576 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:34.294546    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:34.458057 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:34.458028    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"]
Oct 30 06:20:34.463850 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:34.463806    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:20:34.463953 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:34.463905    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:20:34.464041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:34.463951    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:20:34.965566 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:34.965528    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 06:20:34.965786 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:34.965628    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n" podUID="c7295343-bb58-43a6-a460-9b26fb26edc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 06:20:35.171375 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:35.171322    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" event={"ID":"c98c4321-bd50-417b-ae05-8def39068949","Type":"ContainerStarted","Data":"814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0"}
Oct 30 06:20:35.171574 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:35.171434    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" event={"ID":"c98c4321-bd50-417b-ae05-8def39068949","Type":"ContainerStarted","Data":"7934ade880224dcca73c9cc73122691fc2146fb320fabb5704863c61ec291e32"}
Oct 30 06:20:35.282684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:35.282470    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" podStartSLOduration=2.282446773 podStartE2EDuration="2.282446773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:20:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:20:35.251973522 +0000 UTC m=+2938.997258844" watchObservedRunningTime="2026-10-30 06:20:35.282446773 +0000 UTC m=+2939.027732109"
Oct 30 06:20:35.287290 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:35.287255    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"]
Oct 30 06:20:35.287549 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:35.287520    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw" podUID="c8ad12c2-b951-4f84-af2b-424744b62255" containerName="maas-api" containerID="cri-o://aa5398b06228b19b1ce8272bda20a6edd1b128d402f49406e2d0267b589cdf0a" gracePeriod=30
Oct 30 06:20:35.411316 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:35.411270    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw" podUID="c8ad12c2-b951-4f84-af2b-424744b62255" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.138:8443/health\": dial tcp 10.132.0.138:8443: connect: connection refused"
Oct 30 06:20:35.416149 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:35.416112    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:35.982762 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:35.982729    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 06:20:35.983390 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:35.983354    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n" podUID="c7295343-bb58-43a6-a460-9b26fb26edc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 06:20:36.217649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.214026    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:36.217649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.216216    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.179:15021/healthz/ready\": dial tcp 10.132.0.179:15021: connect: connection refused" start-of-body=
Oct 30 06:20:36.217649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.216277    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" podUID="c98c4321-bd50-417b-ae05-8def39068949" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.179:15021/healthz/ready\": dial tcp 10.132.0.179:15021: connect: connection refused"
Oct 30 06:20:36.297874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.294724    5066 generic.go:356] "Generic (PLEG): container finished" podID="c8ad12c2-b951-4f84-af2b-424744b62255" containerID="aa5398b06228b19b1ce8272bda20a6edd1b128d402f49406e2d0267b589cdf0a" exitCode=0
Oct 30 06:20:36.297874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.295484    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw" event={"ID":"c8ad12c2-b951-4f84-af2b-424744b62255","Type":"ContainerDied","Data":"aa5398b06228b19b1ce8272bda20a6edd1b128d402f49406e2d0267b589cdf0a"}
Oct 30 06:20:36.315108 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.315082    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:36.466614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.464409    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:20:36.581620 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.581446    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:36.637721 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.637435    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8ad12c2-b951-4f84-af2b-424744b62255-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ad12c2-b951-4f84-af2b-424744b62255-openshift-service-ca\") pod \"c8ad12c2-b951-4f84-af2b-424744b62255\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") "
Oct 30 06:20:36.638107 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.638084    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8ad12c2-b951-4f84-af2b-424744b62255-kube-api-access-skdzg\" (UniqueName: \"kubernetes.io/projected/c8ad12c2-b951-4f84-af2b-424744b62255-kube-api-access-skdzg\") pod \"c8ad12c2-b951-4f84-af2b-424744b62255\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") "
Oct 30 06:20:36.638265 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.638249    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-maas-api-tls\") pod \"c8ad12c2-b951-4f84-af2b-424744b62255\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") "
Oct 30 06:20:36.639453 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.638400    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-metrics-certs\") pod \"c8ad12c2-b951-4f84-af2b-424744b62255\" (UID: \"c8ad12c2-b951-4f84-af2b-424744b62255\") "
Oct 30 06:20:36.640971 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.640946    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ad12c2-b951-4f84-af2b-424744b62255-openshift-service-ca" pod "c8ad12c2-b951-4f84-af2b-424744b62255" (UID: "c8ad12c2-b951-4f84-af2b-424744b62255"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:20:36.669073 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.669037    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-metrics-certs" pod "c8ad12c2-b951-4f84-af2b-424744b62255" (UID: "c8ad12c2-b951-4f84-af2b-424744b62255"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:20:36.694244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.694163    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8ad12c2-b951-4f84-af2b-424744b62255-kube-api-access-skdzg" pod "c8ad12c2-b951-4f84-af2b-424744b62255" (UID: "c8ad12c2-b951-4f84-af2b-424744b62255"). InnerVolumeSpecName "kube-api-access-skdzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:20:36.695665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.695639    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-maas-api-tls" pod "c8ad12c2-b951-4f84-af2b-424744b62255" (UID: "c8ad12c2-b951-4f84-af2b-424744b62255"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:20:36.743962 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.743920    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skdzg\" (UniqueName: \"kubernetes.io/projected/c8ad12c2-b951-4f84-af2b-424744b62255-kube-api-access-skdzg\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:36.744195 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.744175    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:36.744318 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.744306    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8ad12c2-b951-4f84-af2b-424744b62255-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:36.744416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.744403    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ad12c2-b951-4f84-af2b-424744b62255-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:36.974749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.974696    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 06:20:36.975542 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:36.975032    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n" podUID="c7295343-bb58-43a6-a460-9b26fb26edc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 06:20:37.007242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.007212    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:37.056773 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.056734    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"
Oct 30 06:20:37.091043 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.091008    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.179:15021/healthz/ready\": dial tcp 10.132.0.179:15021: connect: connection refused" start-of-body=
Oct 30 06:20:37.091352 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.091323    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" podUID="c98c4321-bd50-417b-ae05-8def39068949" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.179:15021/healthz/ready\": dial tcp 10.132.0.179:15021: connect: connection refused"
Oct 30 06:20:37.307408 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.307327    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"
Oct 30 06:20:37.307962 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.307548    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw" event={"ID":"c8ad12c2-b951-4f84-af2b-424744b62255","Type":"ContainerDied","Data":"4147cc2358648f556d21e2df1382318989d2875fabcede34088e9bd57e7b0291"}
Oct 30 06:20:37.308134 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.308122    5066 scope.go:122] "RemoveContainer" containerID="aa5398b06228b19b1ce8272bda20a6edd1b128d402f49406e2d0267b589cdf0a"
Oct 30 06:20:37.387632 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.387602    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"]
Oct 30 06:20:37.391304 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.391278    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-03d349-57d58fb66c-6mprw"]
Oct 30 06:20:37.773563 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.773535    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:37.952704 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.950528    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:37.966396 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.966361    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 06:20:37.966564 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:37.966417    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n" podUID="c7295343-bb58-43a6-a460-9b26fb26edc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 06:20:38.070988 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:38.070904    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.179:15021/healthz/ready\": dial tcp 10.132.0.179:15021: connect: connection refused" start-of-body=
Oct 30 06:20:38.070988 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:38.070969    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" podUID="c98c4321-bd50-417b-ae05-8def39068949" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.179:15021/healthz/ready\": dial tcp 10.132.0.179:15021: connect: connection refused"
Oct 30 06:20:38.408751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:38.408710    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8ad12c2-b951-4f84-af2b-424744b62255" path="/var/lib/kubelet/pods/c8ad12c2-b951-4f84-af2b-424744b62255/volumes"
Oct 30 06:20:38.782314 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:38.782224    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:38.971412 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:38.971382    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:38.971852 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:38.971828    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:38.972669 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:38.972650    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:20:39.070915 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:39.070835    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.179:15021/healthz/ready\": dial tcp 10.132.0.179:15021: connect: connection refused" start-of-body=
Oct 30 06:20:39.070915 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:39.070892    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" podUID="c98c4321-bd50-417b-ae05-8def39068949" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.179:15021/healthz/ready\": dial tcp 10.132.0.179:15021: connect: connection refused"
Oct 30 06:20:40.071438 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:40.071403    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.179:15021/healthz/ready\": dial tcp 10.132.0.179:15021: connect: connection refused" start-of-body=
Oct 30 06:20:40.071945 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:40.071460    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" podUID="c98c4321-bd50-417b-ae05-8def39068949" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.179:15021/healthz/ready\": dial tcp 10.132.0.179:15021: connect: connection refused"
Oct 30 06:20:40.758972 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:40.758943    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:41.070792 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:41.070706    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.179:15021/healthz/ready\": dial tcp 10.132.0.179:15021: connect: connection refused" start-of-body=
Oct 30 06:20:41.070792 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:41.070770    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" podUID="c98c4321-bd50-417b-ae05-8def39068949" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.179:15021/healthz/ready\": dial tcp 10.132.0.179:15021: connect: connection refused"
Oct 30 06:20:41.251727 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:41.251694    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99_43685c17-86ec-4839-bb32-7251aba4e869/payload-processing/0.log"
Oct 30 06:20:42.075782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:42.075753    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:42.076230 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:42.076211    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:42.077116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:42.077095    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:20:42.664625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:42.664196    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:43.768646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:43.768571    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:43.932882 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:43.932846    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:44.237476 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:44.218720    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"]
Oct 30 06:20:44.438617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:44.438024    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:44.458612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:44.456075    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw" podUID="d7ff0902-2fa3-4267-b865-3b301828a993" containerName="istio-proxy" containerID="cri-o://30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3" gracePeriod=30
Oct 30 06:20:44.690163 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:44.689073    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:44.867643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:44.867576    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:45.073420 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.072211    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:45.457709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.457672    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:45.628240 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.625904    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:45.687289 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.687250    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"]
Oct 30 06:20:45.688367 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.688336    5066 cpu_manager.go:414] "containerMap: removing container" podUID="c8ad12c2-b951-4f84-af2b-424744b62255" containerName="maas-api"
Oct 30 06:20:45.688367 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.688367    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8ad12c2-b951-4f84-af2b-424744b62255" containerName="maas-api"
Oct 30 06:20:45.688561 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.688549    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8ad12c2-b951-4f84-af2b-424744b62255" containerName="maas-api"
Oct 30 06:20:45.694424 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.694402    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.735724 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.735425    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"]
Oct 30 06:20:45.833370 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.833331    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a6d1b0d-6486-479c-a878-93cb39839013-istiod-ca-cert\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.833654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.833456    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a6d1b0d-6486-479c-a878-93cb39839013-istio-podinfo\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.833654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.833529    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-istio-token\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.833654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.833634    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-certs\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.835216 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.833680    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-credential-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.835216 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.833772    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-envoy\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.835216 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.833879    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.835216 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.833926    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-data\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.835216 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.833967    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd8k9\" (UniqueName: \"kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-kube-api-access-pd8k9\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.942616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.941788    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.942616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.941855    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-data\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.942616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.941903    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd8k9\" (UniqueName: \"kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-kube-api-access-pd8k9\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.942616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.941960    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a6d1b0d-6486-479c-a878-93cb39839013-istiod-ca-cert\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.942616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.942061    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a6d1b0d-6486-479c-a878-93cb39839013-istio-podinfo\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.942616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.942125    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-istio-token\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.942616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.942181    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-certs\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.942616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.942223    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-credential-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.942616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.942310    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-envoy\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.943557 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.943340    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a6d1b0d-6486-479c-a878-93cb39839013-istiod-ca-cert\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.977503 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.976256    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a6d1b0d-6486-479c-a878-93cb39839013-istio-podinfo\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.978726 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.978564    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-data\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.981886 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.981844    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.982073 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.982051    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd8k9\" (UniqueName: \"kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-kube-api-access-pd8k9\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.982208 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.982094    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-certs\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.982332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.982235    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-credential-socket\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:45.992529 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:45.992454    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-envoy\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:46.022141 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:46.021818    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-istio-token\") pod \"e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") " pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:46.204415 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:46.204375    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:46.298267 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:46.298181    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"
Oct 30 06:20:46.310192 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:46.310158    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:46.375567 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:46.375536    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:46.544451 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:46.544416    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:46.658066 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:20:46.658036    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a6d1b0d_6486_479c_a878_93cb39839013.slice/crio-56d435b906fef2ffd0e45731f04482cfcfbfa40a9921305ccc44c24db52b2031 WatchSource:0}: Error finding container 56d435b906fef2ffd0e45731f04482cfcfbfa40a9921305ccc44c24db52b2031: Status 404 returned error can't find the container with id 56d435b906fef2ffd0e45731f04482cfcfbfa40a9921305ccc44c24db52b2031
Oct 30 06:20:46.658974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:46.658858    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"]
Oct 30 06:20:46.661409 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:46.661379    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:20:46.661527 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:46.661465    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:20:46.661527 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:46.661516    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:20:46.749725 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:46.749680    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:46.918493 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:46.918356    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:47.484947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:47.484909    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd" event={"ID":"2a6d1b0d-6486-479c-a878-93cb39839013","Type":"ContainerStarted","Data":"89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7"}
Oct 30 06:20:47.485366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:47.484974    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd" event={"ID":"2a6d1b0d-6486-479c-a878-93cb39839013","Type":"ContainerStarted","Data":"56d435b906fef2ffd0e45731f04482cfcfbfa40a9921305ccc44c24db52b2031"}
Oct 30 06:20:47.551503 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:47.551453    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd" podStartSLOduration=2.55143501 podStartE2EDuration="2.55143501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:20:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:20:47.549079144 +0000 UTC m=+2951.294364487" watchObservedRunningTime="2026-10-30 06:20:47.55143501 +0000 UTC m=+2951.296720341"
Oct 30 06:20:48.310500 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.310466    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:48.312414 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.312389    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.180:15021/healthz/ready\": dial tcp 10.132.0.180:15021: connect: connection refused" start-of-body=
Oct 30 06:20:48.312552 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.312438    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd" podUID="2a6d1b0d-6486-479c-a878-93cb39839013" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.180:15021/healthz/ready\": dial tcp 10.132.0.180:15021: connect: connection refused"
Oct 30 06:20:48.745561 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.745520    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:48.768092 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.768055    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"]
Oct 30 06:20:48.789614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.788988    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:48.791910 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.791349    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"]
Oct 30 06:20:48.899845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.899814    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-data\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:48.900019 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.899866    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-envoy\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:48.900019 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.899993    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-certs\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:48.900097 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.900030    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd898281-6242-49c8-a18b-f8d01d36863f-istio-podinfo\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:48.900130 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.900094    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd898281-6242-49c8-a18b-f8d01d36863f-istiod-ca-cert\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:48.900166 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.900135    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-credential-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:48.900209 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.900194    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n7mg\" (UniqueName: \"kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-kube-api-access-9n7mg\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:48.900244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.900225    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-istio-token\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:48.900276 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:48.900245    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.001662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.001545    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd898281-6242-49c8-a18b-f8d01d36863f-istiod-ca-cert\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.001662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.001625    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-credential-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.001902 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.001674    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n7mg\" (UniqueName: \"kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-kube-api-access-9n7mg\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.001902 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.001699    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-istio-token\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.001902 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.001717    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.001902 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.001876    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-data\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.002128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.001926    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-envoy\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.002128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.001971    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-certs\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.002128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.002008    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd898281-6242-49c8-a18b-f8d01d36863f-istio-podinfo\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.002128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.002073    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-credential-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.002358 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.002171    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-socket\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.002358 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.002255    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-data\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.002462 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.002372    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-certs\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.002462 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.002414    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd898281-6242-49c8-a18b-f8d01d36863f-istiod-ca-cert\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.004497 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.004473    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-envoy\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.004767 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.004751    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd898281-6242-49c8-a18b-f8d01d36863f-istio-podinfo\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.051569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.051540    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n7mg\" (UniqueName: \"kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-kube-api-access-9n7mg\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.052838 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.052810    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-istio-token\") pod \"e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") " pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.154883 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.147984    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:49.310667 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.310542    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.180:15021/healthz/ready\": dial tcp 10.132.0.180:15021: connect: connection refused" start-of-body=
Oct 30 06:20:49.310667 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.310636    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd" podUID="2a6d1b0d-6486-479c-a878-93cb39839013" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.180:15021/healthz/ready\": dial tcp 10.132.0.180:15021: connect: connection refused"
Oct 30 06:20:49.485910 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.485881    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"]
Oct 30 06:20:49.486054 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:20:49.485961    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd898281_6242_49c8_a18b_f8d01d36863f.slice/crio-789282f7abb8b18ad82e38963eaa88a8c14466cd47189239de3a445644951e1c WatchSource:0}: Error finding container 789282f7abb8b18ad82e38963eaa88a8c14466cd47189239de3a445644951e1c: Status 404 returned error can't find the container with id 789282f7abb8b18ad82e38963eaa88a8c14466cd47189239de3a445644951e1c
Oct 30 06:20:49.489451 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.489413    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:20:49.489572 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.489497    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:20:49.489572 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.489540    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:20:49.501892 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.501860    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr" event={"ID":"dd898281-6242-49c8-a18b-f8d01d36863f","Type":"ContainerStarted","Data":"789282f7abb8b18ad82e38963eaa88a8c14466cd47189239de3a445644951e1c"}
Oct 30 06:20:49.682662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.680535    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"]
Oct 30 06:20:49.701717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.701686    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"]
Oct 30 06:20:49.865197 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.865077    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:49.899647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:49.899610    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:20:50.024615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.022582    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-certs\") pod \"d7ff0902-2fa3-4267-b865-3b301828a993\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") "
Oct 30 06:20:50.024615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.022726    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-istio-token\" (UniqueName: \"kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-istio-token\") pod \"d7ff0902-2fa3-4267-b865-3b301828a993\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") "
Oct 30 06:20:50.024615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.022777    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-data\") pod \"d7ff0902-2fa3-4267-b865-3b301828a993\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") "
Oct 30 06:20:50.024615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.022825    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-envoy\") pod \"d7ff0902-2fa3-4267-b865-3b301828a993\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") "
Oct 30 06:20:50.024615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.022844    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-kube-api-access-4wm2w\" (UniqueName: \"kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-kube-api-access-4wm2w\") pod \"d7ff0902-2fa3-4267-b865-3b301828a993\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") "
Oct 30 06:20:50.024615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.022941    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d7ff0902-2fa3-4267-b865-3b301828a993-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ff0902-2fa3-4267-b865-3b301828a993-istio-podinfo\") pod \"d7ff0902-2fa3-4267-b865-3b301828a993\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") "
Oct 30 06:20:50.024615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.022967    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-credential-socket\") pod \"d7ff0902-2fa3-4267-b865-3b301828a993\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") "
Oct 30 06:20:50.024615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.023024    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7ff0902-2fa3-4267-b865-3b301828a993-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ff0902-2fa3-4267-b865-3b301828a993-istiod-ca-cert\") pod \"d7ff0902-2fa3-4267-b865-3b301828a993\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") "
Oct 30 06:20:50.024615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.023079    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-socket\") pod \"d7ff0902-2fa3-4267-b865-3b301828a993\" (UID: \"d7ff0902-2fa3-4267-b865-3b301828a993\") "
Oct 30 06:20:50.069652 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.052782    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7ff0902-2fa3-4267-b865-3b301828a993-istiod-ca-cert" pod "d7ff0902-2fa3-4267-b865-3b301828a993" (UID: "d7ff0902-2fa3-4267-b865-3b301828a993"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:20:50.069652 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.060749    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-data" pod "d7ff0902-2fa3-4267-b865-3b301828a993" (UID: "d7ff0902-2fa3-4267-b865-3b301828a993"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:20:50.069652 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.061314    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-certs" pod "d7ff0902-2fa3-4267-b865-3b301828a993" (UID: "d7ff0902-2fa3-4267-b865-3b301828a993"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:20:50.070185 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.070162    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-credential-socket" pod "d7ff0902-2fa3-4267-b865-3b301828a993" (UID: "d7ff0902-2fa3-4267-b865-3b301828a993"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:20:50.070469 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.070454    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-socket" pod "d7ff0902-2fa3-4267-b865-3b301828a993" (UID: "d7ff0902-2fa3-4267-b865-3b301828a993"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:20:50.073421 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.073387    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7ff0902-2fa3-4267-b865-3b301828a993-istio-podinfo" pod "d7ff0902-2fa3-4267-b865-3b301828a993" (UID: "d7ff0902-2fa3-4267-b865-3b301828a993"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:20:50.074718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.074668    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-istio-token" pod "d7ff0902-2fa3-4267-b865-3b301828a993" (UID: "d7ff0902-2fa3-4267-b865-3b301828a993"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:20:50.075319 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.075294    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-kube-api-access-4wm2w" pod "d7ff0902-2fa3-4267-b865-3b301828a993" (UID: "d7ff0902-2fa3-4267-b865-3b301828a993"). InnerVolumeSpecName "kube-api-access-4wm2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:20:50.078724 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.078499    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-envoy" pod "d7ff0902-2fa3-4267-b865-3b301828a993" (UID: "d7ff0902-2fa3-4267-b865-3b301828a993"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:20:50.133681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.130001    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:50.133681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.130041    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:50.133681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.130054    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:50.133681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.130070    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:50.133681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.130083    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wm2w\" (UniqueName: \"kubernetes.io/projected/d7ff0902-2fa3-4267-b865-3b301828a993-kube-api-access-4wm2w\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:50.133681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.130097    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ff0902-2fa3-4267-b865-3b301828a993-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:50.133681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.130113    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:50.133681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.130127    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ff0902-2fa3-4267-b865-3b301828a993-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:50.133681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.130143    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ff0902-2fa3-4267-b865-3b301828a993-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:20:50.311287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.311249    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.180:15021/healthz/ready\": dial tcp 10.132.0.180:15021: connect: connection refused" start-of-body=
Oct 30 06:20:50.311497 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.311313    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd" podUID="2a6d1b0d-6486-479c-a878-93cb39839013" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.180:15021/healthz/ready\": dial tcp 10.132.0.180:15021: connect: connection refused"
Oct 30 06:20:50.511844 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.511802    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr" event={"ID":"dd898281-6242-49c8-a18b-f8d01d36863f","Type":"ContainerStarted","Data":"d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b"}
Oct 30 06:20:50.513335 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.513310    5066 generic.go:356] "Generic (PLEG): container finished" podID="d7ff0902-2fa3-4267-b865-3b301828a993" containerID="30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3" exitCode=0
Oct 30 06:20:50.513459 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.513352    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"
Oct 30 06:20:50.513459 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.513376    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw" event={"ID":"d7ff0902-2fa3-4267-b865-3b301828a993","Type":"ContainerDied","Data":"30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3"}
Oct 30 06:20:50.513459 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.513413    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw" event={"ID":"d7ff0902-2fa3-4267-b865-3b301828a993","Type":"ContainerDied","Data":"7f3d8fbd6da92c878e7ba980a27b5574b8119ee5d10071b003bda9fcb6e48b44"}
Oct 30 06:20:50.513459 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.513432    5066 scope.go:122] "RemoveContainer" containerID="30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3"
Oct 30 06:20:50.556023 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.555993    5066 scope.go:122] "RemoveContainer" containerID="30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3"
Oct 30 06:20:50.556452 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:20:50.556426    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3\": container with ID starting with 30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3 not found: ID does not exist" containerID="30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3"
Oct 30 06:20:50.556513 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.556470    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3"} err="failed to get container status \"30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3\": rpc error: code = NotFound desc = could not find container \"30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3\": container with ID starting with 30fc73dfb60516014447fc893af14867bfc67cda4682c568559463027634e7b3 not found: ID does not exist"
Oct 30 06:20:50.585204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.585146    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr" podStartSLOduration=2.585133916 podStartE2EDuration="2.585133916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:20:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:20:50.582626621 +0000 UTC m=+2954.327911955" watchObservedRunningTime="2026-10-30 06:20:50.585133916 +0000 UTC m=+2954.330419242"
Oct 30 06:20:50.659329 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.659295    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"]
Oct 30 06:20:50.662856 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.662818    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-03d349-openshift-default-cdb65584c-q9hsw"]
Oct 30 06:20:50.798843 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:50.792046    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:51.163682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:51.157489    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:51.163682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:51.157676    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused" start-of-body=
Oct 30 06:20:51.163682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:51.157725    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr" podUID="dd898281-6242-49c8-a18b-f8d01d36863f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused"
Oct 30 06:20:51.311000 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:51.310970    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.180:15021/healthz/ready\": dial tcp 10.132.0.180:15021: connect: connection refused" start-of-body=
Oct 30 06:20:51.311266 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:51.311232    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd" podUID="2a6d1b0d-6486-479c-a878-93cb39839013" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.180:15021/healthz/ready\": dial tcp 10.132.0.180:15021: connect: connection refused"
Oct 30 06:20:52.157615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:52.157139    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused" start-of-body=
Oct 30 06:20:52.157615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:52.157221    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr" podUID="dd898281-6242-49c8-a18b-f8d01d36863f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused"
Oct 30 06:20:52.310648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:52.310579    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.180:15021/healthz/ready\": dial tcp 10.132.0.180:15021: connect: connection refused" start-of-body=
Oct 30 06:20:52.311063 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:52.310663    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd" podUID="2a6d1b0d-6486-479c-a878-93cb39839013" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.180:15021/healthz/ready\": dial tcp 10.132.0.180:15021: connect: connection refused"
Oct 30 06:20:52.405018 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:52.404984    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7ff0902-2fa3-4267-b865-3b301828a993" path="/var/lib/kubelet/pods/d7ff0902-2fa3-4267-b865-3b301828a993/volumes"
Oct 30 06:20:52.994691 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:52.994648    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:53.157035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:53.156998    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused" start-of-body=
Oct 30 06:20:53.157341 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:53.157309    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr" podUID="dd898281-6242-49c8-a18b-f8d01d36863f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused"
Oct 30 06:20:53.249561 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:53.249463    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:53.311286 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:53.311259    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.180:15021/healthz/ready\": dial tcp 10.132.0.180:15021: connect: connection refused" start-of-body=
Oct 30 06:20:53.311725 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:53.311704    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd" podUID="2a6d1b0d-6486-479c-a878-93cb39839013" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.180:15021/healthz/ready\": dial tcp 10.132.0.180:15021: connect: connection refused"
Oct 30 06:20:53.466193 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:53.466158    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:54.165175 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:54.163114    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused" start-of-body=
Oct 30 06:20:54.165175 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:54.163214    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr" podUID="dd898281-6242-49c8-a18b-f8d01d36863f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused"
Oct 30 06:20:54.317942 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:54.317912    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:54.318428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:54.318402    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:54.319355 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:54.319333    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:20:54.491847 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:54.491763    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:54.705153 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:54.704902    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:54.785413 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:54.785325    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"]
Oct 30 06:20:54.785751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:54.785561    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerName="istio-proxy" containerID="cri-o://b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60" gracePeriod=30
Oct 30 06:20:54.889614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:54.889008    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:55.133995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:55.130941    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:20:55.133995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:55.131036    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:20:55.153603 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:55.153556    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused" start-of-body=
Oct 30 06:20:55.153897 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:55.153852    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr" podUID="dd898281-6242-49c8-a18b-f8d01d36863f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused"
Oct 30 06:20:55.868578 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:55.868539    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:56.157703 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:56.153850    5066 patch_prober.go:29] interesting pod/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused" start-of-body=
Oct 30 06:20:56.157703 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:56.153916    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr" podUID="dd898281-6242-49c8-a18b-f8d01d36863f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused"
Oct 30 06:20:56.249864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:56.249834    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:56.418014 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:56.417935    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:20:57.158447 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:57.158414    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:57.158936 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:57.158921    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:57.159729 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:57.159709    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:20:58.389622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:20:58.389209    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:00.099657 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.099627    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:21:00.117821 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.117788    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-envoy\") pod \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") "
Oct 30 06:21:00.117999 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.117903    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-podinfo\") pod \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") "
Oct 30 06:21:00.117999 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.117934    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istiod-ca-cert\") pod \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") "
Oct 30 06:21:00.118131 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.118015    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-credential-socket\") pod \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") "
Oct 30 06:21:00.118131 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.118084    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-token\" (UniqueName: \"kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-token\") pod \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") "
Oct 30 06:21:00.118249 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.118207    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-kube-api-access-l54ss\" (UniqueName: \"kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-kube-api-access-l54ss\") pod \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") "
Oct 30 06:21:00.118337 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.118317    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-certs\") pod \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") "
Oct 30 06:21:00.118405 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.118371    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-socket\") pod \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") "
Oct 30 06:21:00.118462 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.118433    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-data\") pod \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\" (UID: \"ea38610c-5a57-41c5-bd84-f2c66a5903f6\") "
Oct 30 06:21:00.118634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.118598    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istiod-ca-cert" pod "ea38610c-5a57-41c5-bd84-f2c66a5903f6" (UID: "ea38610c-5a57-41c5-bd84-f2c66a5903f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:21:00.119067 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.119042    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:00.119313 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.119293    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-data" pod "ea38610c-5a57-41c5-bd84-f2c66a5903f6" (UID: "ea38610c-5a57-41c5-bd84-f2c66a5903f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:00.119784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.119707    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-credential-socket" pod "ea38610c-5a57-41c5-bd84-f2c66a5903f6" (UID: "ea38610c-5a57-41c5-bd84-f2c66a5903f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:00.119899 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.119798    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-certs" pod "ea38610c-5a57-41c5-bd84-f2c66a5903f6" (UID: "ea38610c-5a57-41c5-bd84-f2c66a5903f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:00.124426 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.123549    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-socket" pod "ea38610c-5a57-41c5-bd84-f2c66a5903f6" (UID: "ea38610c-5a57-41c5-bd84-f2c66a5903f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:00.124426 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.123689    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-envoy" pod "ea38610c-5a57-41c5-bd84-f2c66a5903f6" (UID: "ea38610c-5a57-41c5-bd84-f2c66a5903f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:00.124426 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.123813    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-token" pod "ea38610c-5a57-41c5-bd84-f2c66a5903f6" (UID: "ea38610c-5a57-41c5-bd84-f2c66a5903f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:21:00.124426 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.123905    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-podinfo" pod "ea38610c-5a57-41c5-bd84-f2c66a5903f6" (UID: "ea38610c-5a57-41c5-bd84-f2c66a5903f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:21:00.126148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.126125    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-kube-api-access-l54ss" pod "ea38610c-5a57-41c5-bd84-f2c66a5903f6" (UID: "ea38610c-5a57-41c5-bd84-f2c66a5903f6"). InnerVolumeSpecName "kube-api-access-l54ss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:21:00.220426 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.220392    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:00.220426 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.220430    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:00.220650 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.220444    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:00.220650 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.220456    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:00.220650 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.220470    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l54ss\" (UniqueName: \"kubernetes.io/projected/ea38610c-5a57-41c5-bd84-f2c66a5903f6-kube-api-access-l54ss\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:00.220650 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.220483    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:00.220650 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.220512    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:00.220650 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.220525    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea38610c-5a57-41c5-bd84-f2c66a5903f6-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:00.255938 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.255854    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:00.624033 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.623992    5066 generic.go:356] "Generic (PLEG): container finished" podID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerID="b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60" exitCode=0
Oct 30 06:21:00.624252 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.624076    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"
Oct 30 06:21:00.624252 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.624082    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" event={"ID":"ea38610c-5a57-41c5-bd84-f2c66a5903f6","Type":"ContainerDied","Data":"b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60"}
Oct 30 06:21:00.624252 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.624176    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx" event={"ID":"ea38610c-5a57-41c5-bd84-f2c66a5903f6","Type":"ContainerDied","Data":"6904f505399dac4435d7e34f6209326142e9b7c54fe0bcfa1f2f9a8e7e279ca9"}
Oct 30 06:21:00.624252 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.624200    5066 scope.go:122] "RemoveContainer" containerID="b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60"
Oct 30 06:21:00.638512 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.638486    5066 scope.go:122] "RemoveContainer" containerID="b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60"
Oct 30 06:21:00.639036 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:21:00.638995    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60\": container with ID starting with b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60 not found: ID does not exist" containerID="b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60"
Oct 30 06:21:00.639143 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.639059    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60"} err="failed to get container status \"b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60\": rpc error: code = NotFound desc = could not find container \"b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60\": container with ID starting with b70b14ca9d4d89708c723c388c20982d07b40986d8dc4f1889f6a1c6249aeb60 not found: ID does not exist"
Oct 30 06:21:00.688707 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.688608    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"]
Oct 30 06:21:00.692884 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:00.692859    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-d1bfce-openshift-default-78bfb84cf-pr4wx"]
Oct 30 06:21:02.039948 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:02.037549    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:02.404280 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:02.404249    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" path="/var/lib/kubelet/pods/ea38610c-5a57-41c5-bd84-f2c66a5903f6/volumes"
Oct 30 06:21:02.639624 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:21:02.638796    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:21:03.964410 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:03.956769    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:04.651318 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:04.651286    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:04.739573 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:04.739538    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main" containerID="cri-o://a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201" gracePeriod=45
Oct 30 06:21:04.790876 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:04.790845    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" containerID="cri-o://8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02" gracePeriod=45
Oct 30 06:21:04.812335 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:04.812303    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:05.133832 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.133802    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 06:21:05.137416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.137394    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 06:21:05.261668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.261094    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f14e50e-4bde-4a37-838c-63b10269fa9e-kube-api-access-6d2p4\" (UniqueName: \"kubernetes.io/projected/6f14e50e-4bde-4a37-838c-63b10269fa9e-kube-api-access-6d2p4\") pod \"6f14e50e-4bde-4a37-838c-63b10269fa9e\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") "
Oct 30 06:21:05.261668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.261203    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-tmp-dir\") pod \"133f5193-04ff-4d67-8515-8e5338675d6c\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") "
Oct 30 06:21:05.261668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.261238    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-model-cache\") pod \"6f14e50e-4bde-4a37-838c-63b10269fa9e\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") "
Oct 30 06:21:05.261668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.261284    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-dshm\") pod \"6f14e50e-4bde-4a37-838c-63b10269fa9e\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") "
Oct 30 06:21:05.261668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.261356    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/133f5193-04ff-4d67-8515-8e5338675d6c-tls-certs\" (UniqueName: \"kubernetes.io/secret/133f5193-04ff-4d67-8515-8e5338675d6c-tls-certs\") pod \"133f5193-04ff-4d67-8515-8e5338675d6c\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") "
Oct 30 06:21:05.261668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.261444    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/133f5193-04ff-4d67-8515-8e5338675d6c-kube-api-access-xjxdh\" (UniqueName: \"kubernetes.io/projected/133f5193-04ff-4d67-8515-8e5338675d6c-kube-api-access-xjxdh\") pod \"133f5193-04ff-4d67-8515-8e5338675d6c\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") "
Oct 30 06:21:05.261668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.261497    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-tmp-dir\") pod \"6f14e50e-4bde-4a37-838c-63b10269fa9e\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") "
Oct 30 06:21:05.261668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.261530    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-home\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-home\") pod \"133f5193-04ff-4d67-8515-8e5338675d6c\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") "
Oct 30 06:21:05.288614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.283713    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-dshm\") pod \"133f5193-04ff-4d67-8515-8e5338675d6c\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") "
Oct 30 06:21:05.288614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.283808    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f14e50e-4bde-4a37-838c-63b10269fa9e-tls-certs\" (UniqueName: \"kubernetes.io/secret/6f14e50e-4bde-4a37-838c-63b10269fa9e-tls-certs\") pod \"6f14e50e-4bde-4a37-838c-63b10269fa9e\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") "
Oct 30 06:21:05.288614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.283881    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-home\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-home\") pod \"6f14e50e-4bde-4a37-838c-63b10269fa9e\" (UID: \"6f14e50e-4bde-4a37-838c-63b10269fa9e\") "
Oct 30 06:21:05.288614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.283987    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-model-cache\") pod \"133f5193-04ff-4d67-8515-8e5338675d6c\" (UID: \"133f5193-04ff-4d67-8515-8e5338675d6c\") "
Oct 30 06:21:05.306622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.293427    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-model-cache" pod "133f5193-04ff-4d67-8515-8e5338675d6c" (UID: "133f5193-04ff-4d67-8515-8e5338675d6c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:05.306622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.293824    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-tmp-dir" pod "6f14e50e-4bde-4a37-838c-63b10269fa9e" (UID: "6f14e50e-4bde-4a37-838c-63b10269fa9e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:05.306622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.303693    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-tmp-dir" pod "133f5193-04ff-4d67-8515-8e5338675d6c" (UID: "133f5193-04ff-4d67-8515-8e5338675d6c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:05.306622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.304116    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-home" pod "6f14e50e-4bde-4a37-838c-63b10269fa9e" (UID: "6f14e50e-4bde-4a37-838c-63b10269fa9e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:05.306622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.305801    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-home" pod "133f5193-04ff-4d67-8515-8e5338675d6c" (UID: "133f5193-04ff-4d67-8515-8e5338675d6c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:05.306622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.305994    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-model-cache" pod "6f14e50e-4bde-4a37-838c-63b10269fa9e" (UID: "6f14e50e-4bde-4a37-838c-63b10269fa9e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:05.392869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.390786    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/133f5193-04ff-4d67-8515-8e5338675d6c-kube-api-access-xjxdh" pod "133f5193-04ff-4d67-8515-8e5338675d6c" (UID: "133f5193-04ff-4d67-8515-8e5338675d6c"). InnerVolumeSpecName "kube-api-access-xjxdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:21:05.395243 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.394466    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:05.396533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.395338    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:05.396533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.395367    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:05.396533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.395402    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:05.396533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.395414    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjxdh\" (UniqueName: \"kubernetes.io/projected/133f5193-04ff-4d67-8515-8e5338675d6c-kube-api-access-xjxdh\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:05.396533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.395433    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:05.396533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.395473    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:05.404380 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.404329    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-dshm" pod "6f14e50e-4bde-4a37-838c-63b10269fa9e" (UID: "6f14e50e-4bde-4a37-838c-63b10269fa9e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:05.404810 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.404785    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f14e50e-4bde-4a37-838c-63b10269fa9e-tls-certs" pod "6f14e50e-4bde-4a37-838c-63b10269fa9e" (UID: "6f14e50e-4bde-4a37-838c-63b10269fa9e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:21:05.412347 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.412298    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/133f5193-04ff-4d67-8515-8e5338675d6c-tls-certs" pod "133f5193-04ff-4d67-8515-8e5338675d6c" (UID: "133f5193-04ff-4d67-8515-8e5338675d6c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:21:05.416207 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.416181    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-dshm" pod "133f5193-04ff-4d67-8515-8e5338675d6c" (UID: "133f5193-04ff-4d67-8515-8e5338675d6c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:05.419721 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.419563    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f14e50e-4bde-4a37-838c-63b10269fa9e-kube-api-access-6d2p4" pod "6f14e50e-4bde-4a37-838c-63b10269fa9e" (UID: "6f14e50e-4bde-4a37-838c-63b10269fa9e"). InnerVolumeSpecName "kube-api-access-6d2p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:21:05.497951 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.497811    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/133f5193-04ff-4d67-8515-8e5338675d6c-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:05.497951 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.497844    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f14e50e-4bde-4a37-838c-63b10269fa9e-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:05.497951 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.497861    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d2p4\" (UniqueName: \"kubernetes.io/projected/6f14e50e-4bde-4a37-838c-63b10269fa9e-kube-api-access-6d2p4\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:05.497951 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.497875    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f14e50e-4bde-4a37-838c-63b10269fa9e-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:05.497951 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.497890    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/133f5193-04ff-4d67-8515-8e5338675d6c-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:05.702681 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.702637    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:05.727148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.727116    5066 generic.go:356] "Generic (PLEG): container finished" podID="133f5193-04ff-4d67-8515-8e5338675d6c" containerID="a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201" exitCode=0
Oct 30 06:21:05.727345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.727186    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"
Oct 30 06:21:05.727345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.727190    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" event={"ID":"133f5193-04ff-4d67-8515-8e5338675d6c","Type":"ContainerDied","Data":"a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201"}
Oct 30 06:21:05.727345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.727306    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj" event={"ID":"133f5193-04ff-4d67-8515-8e5338675d6c","Type":"ContainerDied","Data":"38202af789d69b9b9dfc7bd5c03ca945766a4bbda36aecdc5183d32230a98d71"}
Oct 30 06:21:05.727345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.727331    5066 scope.go:122] "RemoveContainer" containerID="a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201"
Oct 30 06:21:05.729277 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.729249    5066 generic.go:356] "Generic (PLEG): container finished" podID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerID="8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02" exitCode=0
Oct 30 06:21:05.729393 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.729328    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" event={"ID":"6f14e50e-4bde-4a37-838c-63b10269fa9e","Type":"ContainerDied","Data":"8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02"}
Oct 30 06:21:05.729393 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.729362    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" event={"ID":"6f14e50e-4bde-4a37-838c-63b10269fa9e","Type":"ContainerDied","Data":"41201c01e46d9befd94ab9c934510c6bbb7e75a93dda96982acc499fb0677e28"}
Oct 30 06:21:05.729492 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.729426    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"
Oct 30 06:21:05.740263 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.740237    5066 scope.go:122] "RemoveContainer" containerID="a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201"
Oct 30 06:21:05.740748 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:21:05.740719    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201\": container with ID starting with a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201 not found: ID does not exist" containerID="a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201"
Oct 30 06:21:05.740873 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.740760    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201"} err="failed to get container status \"a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201\": rpc error: code = NotFound desc = could not find container \"a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201\": container with ID starting with a9a6a9d45bbdcbf487e3dfefa3ebabd30ea4a229a77bc48385e55d54bd6f4201 not found: ID does not exist"
Oct 30 06:21:05.740873 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.740785    5066 scope.go:122] "RemoveContainer" containerID="8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02"
Oct 30 06:21:05.754002 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.753876    5066 scope.go:122] "RemoveContainer" containerID="8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02"
Oct 30 06:21:05.754302 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:21:05.754279    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02\": container with ID starting with 8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02 not found: ID does not exist" containerID="8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02"
Oct 30 06:21:05.754387 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.754312    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02"} err="failed to get container status \"8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02\": rpc error: code = NotFound desc = could not find container \"8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02\": container with ID starting with 8f5b310d4cb2ed91d29f39a28f5e98c8cb79ad4c2874b75ccf7f177294e1bb02 not found: ID does not exist"
Oct 30 06:21:05.793168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.793136    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"]
Oct 30 06:21:05.831989 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.831954    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-197e9b/distinct-2-w0-197e9b-kserve-75f486cf55-cghpj"]
Oct 30 06:21:05.882605 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.882555    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"]
Oct 30 06:21:05.889402 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.889358    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j"]
Oct 30 06:21:05.906388 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:05.906346    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:06.077802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:06.077719    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:06.119657 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:06.119613    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-197e9b/simulator-w0-197e9b-kserve-b658b89f6-mmp4j" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 06:21:06.249607 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:06.249549    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:06.414932 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:06.414898    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" path="/var/lib/kubelet/pods/133f5193-04ff-4d67-8515-8e5338675d6c/volumes"
Oct 30 06:21:06.415870 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:06.415848    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" path="/var/lib/kubelet/pods/6f14e50e-4bde-4a37-838c-63b10269fa9e/volumes"
Oct 30 06:21:06.477461 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:06.477420    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:06.653790 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:06.653752    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:06.826674 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:06.826278    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:07.141695 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:07.141575    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:08.681612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:08.681052    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:10.561612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:10.557699    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:12.465614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:12.465246    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:13.721463 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:13.721422    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:13.906827 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:13.906788    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:14.082939 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:14.082849    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:14.823732 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:14.822554    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:15.538482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:15.538422    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:16.746273 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:16.746227    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:18.480989 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:18.479700    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:20.523236 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:20.505698    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:23.054438 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:23.054403    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:24.918472 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:24.918421    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:26.746050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:26.745175    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:26.919330 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:26.919296    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:28.375279 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:28.375249    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:28.972488 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:28.972448    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:30.893740 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:30.893702    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:32.555303 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:21:32.555267    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:21:32.757221 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:32.757187    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:33.458215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:33.458185    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:34.411738 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:34.411708    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:34.577944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:34.577897    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:35.150133 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:35.150101    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:36.274224 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:36.274176    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:36.916866 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:36.916830    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:21:36.933251 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:36.933229    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:21:38.162566 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:38.161337    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:39.583873 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:39.583828    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"]
Oct 30 06:21:39.672424 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:39.671689    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:40.367365 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:40.367308    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:41.338260 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:41.338230    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:41.513428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:41.513363    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:41.980582 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:41.980550    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:43.024723 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.024687    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"]
Oct 30 06:21:43.025489 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.025458    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45" podUID="7bb61813-598d-4817-bfd5-a3b539e195fe" containerName="maas-api" containerID="cri-o://43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1" gracePeriod=30
Oct 30 06:21:43.163313 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.163277    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:43.477536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.477419    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 06:21:43.565496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.565444    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bb61813-598d-4817-bfd5-a3b539e195fe-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bb61813-598d-4817-bfd5-a3b539e195fe-openshift-service-ca\") pod \"7bb61813-598d-4817-bfd5-a3b539e195fe\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") "
Oct 30 06:21:43.588063 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.588035    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-maas-api-tls\") pod \"7bb61813-598d-4817-bfd5-a3b539e195fe\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") "
Oct 30 06:21:43.601032 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.588857    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-metrics-certs\") pod \"7bb61813-598d-4817-bfd5-a3b539e195fe\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") "
Oct 30 06:21:43.616615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.607696    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bb61813-598d-4817-bfd5-a3b539e195fe-kube-api-access-f795n\" (UniqueName: \"kubernetes.io/projected/7bb61813-598d-4817-bfd5-a3b539e195fe-kube-api-access-f795n\") pod \"7bb61813-598d-4817-bfd5-a3b539e195fe\" (UID: \"7bb61813-598d-4817-bfd5-a3b539e195fe\") "
Oct 30 06:21:43.616615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.567408    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bb61813-598d-4817-bfd5-a3b539e195fe-openshift-service-ca" pod "7bb61813-598d-4817-bfd5-a3b539e195fe" (UID: "7bb61813-598d-4817-bfd5-a3b539e195fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:21:43.616615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.600536    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-metrics-certs" pod "7bb61813-598d-4817-bfd5-a3b539e195fe" (UID: "7bb61813-598d-4817-bfd5-a3b539e195fe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:21:43.672691 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.644719    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-maas-api-tls" pod "7bb61813-598d-4817-bfd5-a3b539e195fe" (UID: "7bb61813-598d-4817-bfd5-a3b539e195fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:21:43.672949 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.667207    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb61813-598d-4817-bfd5-a3b539e195fe-kube-api-access-f795n" pod "7bb61813-598d-4817-bfd5-a3b539e195fe" (UID: "7bb61813-598d-4817-bfd5-a3b539e195fe"). InnerVolumeSpecName "kube-api-access-f795n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:21:43.758617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.732923    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:43.758937 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.758911    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f795n\" (UniqueName: \"kubernetes.io/projected/7bb61813-598d-4817-bfd5-a3b539e195fe-kube-api-access-f795n\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:43.759050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.759036    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bb61813-598d-4817-bfd5-a3b539e195fe-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:43.759147 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.759133    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7bb61813-598d-4817-bfd5-a3b539e195fe-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:43.931117 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:43.931013    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:44.050628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.047666    5066 generic.go:356] "Generic (PLEG): container finished" podID="7bb61813-598d-4817-bfd5-a3b539e195fe" containerID="43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1" exitCode=0
Oct 30 06:21:44.050628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.047851    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45" event={"ID":"7bb61813-598d-4817-bfd5-a3b539e195fe","Type":"ContainerDied","Data":"43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1"}
Oct 30 06:21:44.050628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.047907    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45" event={"ID":"7bb61813-598d-4817-bfd5-a3b539e195fe","Type":"ContainerDied","Data":"f16854b591a2320e36ace5656e34d5f5bea13b77e2696d163135601ca5c9b05a"}
Oct 30 06:21:44.050628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.047937    5066 scope.go:122] "RemoveContainer" containerID="43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1"
Oct 30 06:21:44.050628 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.048128    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"
Oct 30 06:21:44.078372 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.078245    5066 scope.go:122] "RemoveContainer" containerID="43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1"
Oct 30 06:21:44.078717 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:21:44.078642    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1\": container with ID starting with 43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1 not found: ID does not exist" containerID="43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1"
Oct 30 06:21:44.078717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.078684    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1"} err="failed to get container status \"43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1\": rpc error: code = NotFound desc = could not find container \"43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1\": container with ID starting with 43159f6ce50065b46b78f2f20524e5b98ace54504f820c62681667dc25f7c0d1 not found: ID does not exist"
Oct 30 06:21:44.104278 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.104246    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:44.115735 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.115706    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"]
Oct 30 06:21:44.167667 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.157204    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-477f6b-78c65f767-lvl45"]
Oct 30 06:21:44.271798 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.271697    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:44.403373 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.403337    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bb61813-598d-4817-bfd5-a3b539e195fe" path="/var/lib/kubelet/pods/7bb61813-598d-4817-bfd5-a3b539e195fe/volumes"
Oct 30 06:21:44.561569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:44.561487    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:46.205175 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:46.204983    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:48.351751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:48.351715    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:48.708837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:48.708753    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:48.870616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:48.870572    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:49.073571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:49.073497    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:49.363675 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:49.335110    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:49.605572 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:49.605533    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:49.781339 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:49.781303    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:50.244616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:50.244453    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:50.593717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:50.593535    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:50.894249 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:50.894110    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:53.020411 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:53.020380    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:53.187478 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:53.187445    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:53.429643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:53.420614    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:53.657635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:53.657600    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:53.860434 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:53.860353    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:54.633800 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.633761    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b" podUID="8e98b564-3662-481a-acc0-125897de5903" containerName="main" containerID="cri-o://516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af" gracePeriod=45
Oct 30 06:21:54.654454 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.654332    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:54.750103 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.750066    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b" podUID="8e98b564-3662-481a-acc0-125897de5903" containerName="main" probeResult="failure" output="Get \"https://10.132.0.171:8000/ready\": dial tcp 10.132.0.171:8000: connect: connection refused"
Oct 30 06:21:54.915684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.915661    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:21:54.948054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.948024    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-home\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-home\") pod \"8e98b564-3662-481a-acc0-125897de5903\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") "
Oct 30 06:21:54.948224 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.948198    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e98b564-3662-481a-acc0-125897de5903-tls-certs\" (UniqueName: \"kubernetes.io/secret/8e98b564-3662-481a-acc0-125897de5903-tls-certs\") pod \"8e98b564-3662-481a-acc0-125897de5903\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") "
Oct 30 06:21:54.948290 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.948231    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-model-cache\") pod \"8e98b564-3662-481a-acc0-125897de5903\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") "
Oct 30 06:21:54.948359 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.948317    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-home" pod "8e98b564-3662-481a-acc0-125897de5903" (UID: "8e98b564-3662-481a-acc0-125897de5903"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:54.948473 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.948438    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-dshm\") pod \"8e98b564-3662-481a-acc0-125897de5903\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") "
Oct 30 06:21:54.948565 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.948481    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-tmp-dir\") pod \"8e98b564-3662-481a-acc0-125897de5903\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") "
Oct 30 06:21:54.948565 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.948526    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-model-cache" pod "8e98b564-3662-481a-acc0-125897de5903" (UID: "8e98b564-3662-481a-acc0-125897de5903"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:54.948565 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.948544    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e98b564-3662-481a-acc0-125897de5903-kube-api-access-qdnzt\" (UniqueName: \"kubernetes.io/projected/8e98b564-3662-481a-acc0-125897de5903-kube-api-access-qdnzt\") pod \"8e98b564-3662-481a-acc0-125897de5903\" (UID: \"8e98b564-3662-481a-acc0-125897de5903\") "
Oct 30 06:21:54.948809 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.948787    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-tmp-dir" pod "8e98b564-3662-481a-acc0-125897de5903" (UID: "8e98b564-3662-481a-acc0-125897de5903"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:54.949353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.949335    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:54.949353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.949355    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:54.949539 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.949365    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:54.950947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.950916    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e98b564-3662-481a-acc0-125897de5903-tls-certs" pod "8e98b564-3662-481a-acc0-125897de5903" (UID: "8e98b564-3662-481a-acc0-125897de5903"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:21:54.951062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.951001    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-dshm" pod "8e98b564-3662-481a-acc0-125897de5903" (UID: "8e98b564-3662-481a-acc0-125897de5903"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:21:54.951062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:54.951010    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e98b564-3662-481a-acc0-125897de5903-kube-api-access-qdnzt" pod "8e98b564-3662-481a-acc0-125897de5903" (UID: "8e98b564-3662-481a-acc0-125897de5903"). InnerVolumeSpecName "kube-api-access-qdnzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:21:55.051120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:55.051089    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e98b564-3662-481a-acc0-125897de5903-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:55.051120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:55.051117    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdnzt\" (UniqueName: \"kubernetes.io/projected/8e98b564-3662-481a-acc0-125897de5903-kube-api-access-qdnzt\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:55.051120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:55.051127    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e98b564-3662-481a-acc0-125897de5903-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:21:55.184350 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:55.184316    5066 generic.go:356] "Generic (PLEG): container finished" podID="8e98b564-3662-481a-acc0-125897de5903" containerID="516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af" exitCode=0
Oct 30 06:21:55.184536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:55.184362    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"
Oct 30 06:21:55.184536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:55.184397    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b" event={"ID":"8e98b564-3662-481a-acc0-125897de5903","Type":"ContainerDied","Data":"516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af"}
Oct 30 06:21:55.184536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:55.184443    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b" event={"ID":"8e98b564-3662-481a-acc0-125897de5903","Type":"ContainerDied","Data":"9dc0b24e6cd4eadd36378aa5729e594daf7ee6d702b21e90dcfacd8df0273d70"}
Oct 30 06:21:55.184536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:55.184463    5066 scope.go:122] "RemoveContainer" containerID="516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af"
Oct 30 06:21:55.196005 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:55.195979    5066 scope.go:122] "RemoveContainer" containerID="516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af"
Oct 30 06:21:55.196284 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:21:55.196263    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af\": container with ID starting with 516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af not found: ID does not exist" containerID="516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af"
Oct 30 06:21:55.196326 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:55.196294    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af"} err="failed to get container status \"516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af\": rpc error: code = NotFound desc = could not find container \"516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af\": container with ID starting with 516079dd3a25cb59702eb1c277067229bd928ac8ae0deb5e6e98868dc44c98af not found: ID does not exist"
Oct 30 06:21:55.250717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:55.250674    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"]
Oct 30 06:21:55.264208 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:55.264179    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w1-0f980e/rate-test-model-0f980e-kserve-59b67f5ccc-k9f2b"]
Oct 30 06:21:56.156636 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:56.156567    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:56.405400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:56.405358    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e98b564-3662-481a-acc0-125897de5903" path="/var/lib/kubelet/pods/8e98b564-3662-481a-acc0-125897de5903/volumes"
Oct 30 06:21:57.856617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:57.856496    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:21:59.550110 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:59.550080    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99_43685c17-86ec-4839-bb32-7251aba4e869/payload-processing/0.log"
Oct 30 06:21:59.743614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:59.741880    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99_43685c17-86ec-4839-bb32-7251aba4e869/payload-processing/0.log"
Oct 30 06:21:59.822069 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:21:59.821991    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:00.339092 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:00.339056    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:00.508178 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:00.508150    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:01.217102 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.217071    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"]
Oct 30 06:22:01.217532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.217267    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk" podUID="191c6961-a046-467d-b8df-613be7c5dcb6" containerName="maas-api" containerID="cri-o://c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d" gracePeriod=30
Oct 30 06:22:01.340051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.340001    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:01.414647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.409036    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"]
Oct 30 06:22:01.414647 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.409353    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg" podUID="bf77c510-85ab-4b5c-a7a3-a62d388ab023" containerName="payload-processing" containerID="cri-o://bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009" gracePeriod=30
Oct 30 06:22:01.545750 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.545327    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:01.599983 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.599946    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:22:01.693777 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.693743    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-maas-api-tls\") pod \"191c6961-a046-467d-b8df-613be7c5dcb6\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") "
Oct 30 06:22:01.694122 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.694098    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/191c6961-a046-467d-b8df-613be7c5dcb6-kube-api-access-6n46b\" (UniqueName: \"kubernetes.io/projected/191c6961-a046-467d-b8df-613be7c5dcb6-kube-api-access-6n46b\") pod \"191c6961-a046-467d-b8df-613be7c5dcb6\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") "
Oct 30 06:22:01.694265 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.694251    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/191c6961-a046-467d-b8df-613be7c5dcb6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/191c6961-a046-467d-b8df-613be7c5dcb6-openshift-service-ca\") pod \"191c6961-a046-467d-b8df-613be7c5dcb6\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") "
Oct 30 06:22:01.694423 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.694409    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-metrics-certs\") pod \"191c6961-a046-467d-b8df-613be7c5dcb6\" (UID: \"191c6961-a046-467d-b8df-613be7c5dcb6\") "
Oct 30 06:22:01.694892 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.694856    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/191c6961-a046-467d-b8df-613be7c5dcb6-openshift-service-ca" pod "191c6961-a046-467d-b8df-613be7c5dcb6" (UID: "191c6961-a046-467d-b8df-613be7c5dcb6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:01.695492 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.695469    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/191c6961-a046-467d-b8df-613be7c5dcb6-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:01.699821 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.699777    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-maas-api-tls" pod "191c6961-a046-467d-b8df-613be7c5dcb6" (UID: "191c6961-a046-467d-b8df-613be7c5dcb6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:22:01.700683 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.700656    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-metrics-certs" pod "191c6961-a046-467d-b8df-613be7c5dcb6" (UID: "191c6961-a046-467d-b8df-613be7c5dcb6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:22:01.700944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.700922    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/191c6961-a046-467d-b8df-613be7c5dcb6-kube-api-access-6n46b" pod "191c6961-a046-467d-b8df-613be7c5dcb6" (UID: "191c6961-a046-467d-b8df-613be7c5dcb6"). InnerVolumeSpecName "kube-api-access-6n46b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:01.719152 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.719121    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:01.753012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.752983    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"
Oct 30 06:22:01.834723 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.827311    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:01.902617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.858651    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n46b\" (UniqueName: \"kubernetes.io/projected/191c6961-a046-467d-b8df-613be7c5dcb6-kube-api-access-6n46b\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:01.902617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.858687    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/191c6961-a046-467d-b8df-613be7c5dcb6-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:01.960009 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.959962    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf77c510-85ab-4b5c-a7a3-a62d388ab023-kube-api-access-xcm4t\" (UniqueName: \"kubernetes.io/projected/bf77c510-85ab-4b5c-a7a3-a62d388ab023-kube-api-access-xcm4t\") pod \"bf77c510-85ab-4b5c-a7a3-a62d388ab023\" (UID: \"bf77c510-85ab-4b5c-a7a3-a62d388ab023\") "
Oct 30 06:22:01.960351 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.960327    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf77c510-85ab-4b5c-a7a3-a62d388ab023-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf77c510-85ab-4b5c-a7a3-a62d388ab023-plugins-config-volume\") pod \"bf77c510-85ab-4b5c-a7a3-a62d388ab023\" (UID: \"bf77c510-85ab-4b5c-a7a3-a62d388ab023\") "
Oct 30 06:22:01.961538 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.961437    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf77c510-85ab-4b5c-a7a3-a62d388ab023-plugins-config-volume" pod "bf77c510-85ab-4b5c-a7a3-a62d388ab023" (UID: "bf77c510-85ab-4b5c-a7a3-a62d388ab023"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:01.970537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:01.969816    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf77c510-85ab-4b5c-a7a3-a62d388ab023-kube-api-access-xcm4t" pod "bf77c510-85ab-4b5c-a7a3-a62d388ab023" (UID: "bf77c510-85ab-4b5c-a7a3-a62d388ab023"). InnerVolumeSpecName "kube-api-access-xcm4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:02.062222 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.062183    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcm4t\" (UniqueName: \"kubernetes.io/projected/bf77c510-85ab-4b5c-a7a3-a62d388ab023-kube-api-access-xcm4t\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:02.062222 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.062217    5066 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf77c510-85ab-4b5c-a7a3-a62d388ab023-plugins-config-volume\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:02.282314 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.282280    5066 generic.go:356] "Generic (PLEG): container finished" podID="bf77c510-85ab-4b5c-a7a3-a62d388ab023" containerID="bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009" exitCode=0
Oct 30 06:22:02.282844 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.282323    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"
Oct 30 06:22:02.282844 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.282373    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg" event={"ID":"bf77c510-85ab-4b5c-a7a3-a62d388ab023","Type":"ContainerDied","Data":"bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009"}
Oct 30 06:22:02.282844 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.282424    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg" event={"ID":"bf77c510-85ab-4b5c-a7a3-a62d388ab023","Type":"ContainerDied","Data":"89b08c34cec654d63ff886ad6df4e6e0d37a72a3fa303acff6a29a442046679e"}
Oct 30 06:22:02.282844 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.282451    5066 scope.go:122] "RemoveContainer" containerID="bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009"
Oct 30 06:22:02.286749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.286723    5066 generic.go:356] "Generic (PLEG): container finished" podID="191c6961-a046-467d-b8df-613be7c5dcb6" containerID="c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d" exitCode=0
Oct 30 06:22:02.286851 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.286759    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk" event={"ID":"191c6961-a046-467d-b8df-613be7c5dcb6","Type":"ContainerDied","Data":"c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d"}
Oct 30 06:22:02.286851 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.286770    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"
Oct 30 06:22:02.286851 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.286802    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk" event={"ID":"191c6961-a046-467d-b8df-613be7c5dcb6","Type":"ContainerDied","Data":"021a0dd5516b2323882948ccc8f726eb7e188ec9a7c3e5778925c16bdb3cfff4"}
Oct 30 06:22:02.298467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.298447    5066 scope.go:122] "RemoveContainer" containerID="bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009"
Oct 30 06:22:02.298768 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:02.298750    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009\": container with ID starting with bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009 not found: ID does not exist" containerID="bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009"
Oct 30 06:22:02.298829 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.298788    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009"} err="failed to get container status \"bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009\": rpc error: code = NotFound desc = could not find container \"bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009\": container with ID starting with bb0e5931937460316a71874a6b3914947418151251ab6bd4aa7cb7d7ba2d0009 not found: ID does not exist"
Oct 30 06:22:02.298829 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.298807    5066 scope.go:122] "RemoveContainer" containerID="c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d"
Oct 30 06:22:02.308483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.308462    5066 scope.go:122] "RemoveContainer" containerID="c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d"
Oct 30 06:22:02.308920 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:02.308900    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d\": container with ID starting with c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d not found: ID does not exist" containerID="c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d"
Oct 30 06:22:02.308995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.308929    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d"} err="failed to get container status \"c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d\": rpc error: code = NotFound desc = could not find container \"c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d\": container with ID starting with c0088bfcb6afc4c5bd82507b1413762b63467a7fafa31db7b7f765fcc54e790d not found: ID does not exist"
Oct 30 06:22:02.326654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.326627    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:02.346725 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.346697    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"]
Oct 30 06:22:02.362044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.362013    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-5deb36-6c7b7c955c-g8rkk"]
Oct 30 06:22:02.403974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.403941    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="191c6961-a046-467d-b8df-613be7c5dcb6" path="/var/lib/kubelet/pods/191c6961-a046-467d-b8df-613be7c5dcb6/volumes"
Oct 30 06:22:02.427178 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.427149    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"]
Oct 30 06:22:02.430517 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.430485    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5deb36-5b9855cb7d-wbhsg"]
Oct 30 06:22:02.492420 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:02.492307    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:02.553379 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:02.553348    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:22:03.030117 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:03.030089    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:04.403155 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:04.403120    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf77c510-85ab-4b5c-a7a3-a62d388ab023" path="/var/lib/kubelet/pods/bf77c510-85ab-4b5c-a7a3-a62d388ab023/volumes"
Oct 30 06:22:04.419161 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:04.419132    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:04.578447 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:04.578418    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:05.951235 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:05.951197    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:06.528654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:06.525008    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:06.692540 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:06.692497    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:07.165852 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:07.165823    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:08.859211 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:08.859175    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:09.424876 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:09.424844    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:09.598408 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:09.598364    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:09.739676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:09.731391    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"]
Oct 30 06:22:09.739676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:09.739557    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n" podUID="c7295343-bb58-43a6-a460-9b26fb26edc0" containerName="istio-proxy" containerID="cri-o://44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d" gracePeriod=30
Oct 30 06:22:09.793638 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:09.792470    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"]
Oct 30 06:22:09.793638 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:09.792738    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd" podUID="2a6d1b0d-6486-479c-a878-93cb39839013" containerName="istio-proxy" containerID="cri-o://89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7" gracePeriod=30
Oct 30 06:22:09.800796 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:09.796046    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:09.827181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:09.827147    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"]
Oct 30 06:22:09.827688 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:09.827653    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc" podUID="8a1d2da0-194d-44b7-94ad-c49bd3cb2451" containerName="istio-proxy" containerID="cri-o://5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5" gracePeriod=30
Oct 30 06:22:10.036909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:10.034582    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:10.262278 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:10.262242    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:10.868000 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:10.867930    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:11.109542 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:11.109506    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:11.290046 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:11.290014    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:12.952377 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:12.952344    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:13.561096 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:13.561064    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:13.773369 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:13.773324    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:13.975283 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:13.975254    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:14.222368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.222331    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:14.614180 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.614140    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:14.736192 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736157    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"]
Oct 30 06:22:14.736885 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736869    5066 cpu_manager.go:414] "containerMap: removing container" podUID="7bb61813-598d-4817-bfd5-a3b539e195fe" containerName="maas-api"
Oct 30 06:22:14.736954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736887    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bb61813-598d-4817-bfd5-a3b539e195fe" containerName="maas-api"
Oct 30 06:22:14.736954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736901    5066 cpu_manager.go:414] "containerMap: removing container" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main"
Oct 30 06:22:14.736954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736906    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main"
Oct 30 06:22:14.736954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736916    5066 cpu_manager.go:414] "containerMap: removing container" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerName="istio-proxy"
Oct 30 06:22:14.736954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736921    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerName="istio-proxy"
Oct 30 06:22:14.736954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736931    5066 cpu_manager.go:414] "containerMap: removing container" podUID="191c6961-a046-467d-b8df-613be7c5dcb6" containerName="maas-api"
Oct 30 06:22:14.736954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736935    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="191c6961-a046-467d-b8df-613be7c5dcb6" containerName="maas-api"
Oct 30 06:22:14.736954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736945    5066 cpu_manager.go:414] "containerMap: removing container" podUID="d7ff0902-2fa3-4267-b865-3b301828a993" containerName="istio-proxy"
Oct 30 06:22:14.736954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736949    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7ff0902-2fa3-4267-b865-3b301828a993" containerName="istio-proxy"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736962    5066 cpu_manager.go:414] "containerMap: removing container" podUID="8e98b564-3662-481a-acc0-125897de5903" containerName="main"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736968    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e98b564-3662-481a-acc0-125897de5903" containerName="main"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736976    5066 cpu_manager.go:414] "containerMap: removing container" podUID="bf77c510-85ab-4b5c-a7a3-a62d388ab023" containerName="payload-processing"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.736980    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf77c510-85ab-4b5c-a7a3-a62d388ab023" containerName="payload-processing"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.737003    5066 cpu_manager.go:414] "containerMap: removing container" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.737008    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.737102    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7ff0902-2fa3-4267-b865-3b301828a993" containerName="istio-proxy"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.737112    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e98b564-3662-481a-acc0-125897de5903" containerName="main"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.737129    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf77c510-85ab-4b5c-a7a3-a62d388ab023" containerName="payload-processing"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.737138    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="133f5193-04ff-4d67-8515-8e5338675d6c" containerName="main"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.737146    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bb61813-598d-4817-bfd5-a3b539e195fe" containerName="maas-api"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.737152    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f14e50e-4bde-4a37-838c-63b10269fa9e" containerName="main"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.737158    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea38610c-5a57-41c5-bd84-f2c66a5903f6" containerName="istio-proxy"
Oct 30 06:22:14.737259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.737164    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="191c6961-a046-467d-b8df-613be7c5dcb6" containerName="maas-api"
Oct 30 06:22:14.740868 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.740853    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.772864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.772822    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"]
Oct 30 06:22:14.794101 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.794064    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:14.847772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.846616    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-credential-socket\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.847772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.846657    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-token\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.847772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.846682    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-data\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.847772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.846707    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-podinfo\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.847772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.846968    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-envoy\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.847772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.847029    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjjcs\" (UniqueName: \"kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-kube-api-access-vjjcs\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.847772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.847167    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-certs\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.847772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.847194    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eda6dbe1-e406-4d55-a0e2-1c532edda522-istiod-ca-cert\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.847772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.847343    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-socket\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.889346 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.888771    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"]
Oct 30 06:22:14.889346 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.889014    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986" podUID="f575bda0-bead-41dc-a5e5-d633cad8e59d" containerName="istio-proxy" containerID="cri-o://c52bc9b8a2fca45f7cb8b8781d9d02e4f860e6b2d95c3aeb00831d57924f7c74" gracePeriod=30
Oct 30 06:22:14.948756 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.948724    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-envoy\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.948880 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.948795    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjjcs\" (UniqueName: \"kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-kube-api-access-vjjcs\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.948880 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.948862    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-certs\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.949020 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.948887    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eda6dbe1-e406-4d55-a0e2-1c532edda522-istiod-ca-cert\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.949020 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.948946    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-socket\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.949142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.949046    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-credential-socket\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.949142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.949073    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-token\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.949142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.949097    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-data\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.949142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.949133    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-podinfo\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.950186 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.950083    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-credential-socket\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.950186 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.950139    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eda6dbe1-e406-4d55-a0e2-1c532edda522-istiod-ca-cert\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.950383 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.950355    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-data\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.950507 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.950488    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-socket\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.950783 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.950764    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-certs\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.955010 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.954984    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-podinfo\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.955245 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.955224    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-envoy\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.989332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.989287    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-token\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:14.990516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.990478    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:14.996128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:14.996097    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjjcs\" (UniqueName: \"kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-kube-api-access-vjjcs\") pod \"e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") " pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:15.060375 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.060240    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:15.169612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.159795    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:15.279035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.279003    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:22:15.351370 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.351337    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:22:15.461698 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.461669    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:22:15.463301 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463269    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-token\" (UniqueName: \"kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-token\") pod \"c7295343-bb58-43a6-a460-9b26fb26edc0\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") "
Oct 30 06:22:15.463520 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463497    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-certs\") pod \"c7295343-bb58-43a6-a460-9b26fb26edc0\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") "
Oct 30 06:22:15.463638 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463563    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-socket\") pod \"c7295343-bb58-43a6-a460-9b26fb26edc0\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") "
Oct 30 06:22:15.463718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463634    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-kube-api-access-xhsrx\" (UniqueName: \"kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-kube-api-access-xhsrx\") pod \"c7295343-bb58-43a6-a460-9b26fb26edc0\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") "
Oct 30 06:22:15.463718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463669    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-podinfo\") pod \"c7295343-bb58-43a6-a460-9b26fb26edc0\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") "
Oct 30 06:22:15.463718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463706    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-kube-api-access-kmwbk\" (UniqueName: \"kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-kube-api-access-kmwbk\") pod \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") "
Oct 30 06:22:15.463884 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463739    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-credential-socket\") pod \"c7295343-bb58-43a6-a460-9b26fb26edc0\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") "
Oct 30 06:22:15.463884 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463779    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-token\" (UniqueName: \"kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-token\") pod \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") "
Oct 30 06:22:15.463884 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463808    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7295343-bb58-43a6-a460-9b26fb26edc0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7295343-bb58-43a6-a460-9b26fb26edc0-istiod-ca-cert\") pod \"c7295343-bb58-43a6-a460-9b26fb26edc0\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") "
Oct 30 06:22:15.463884 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463829    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-podinfo\") pod \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") "
Oct 30 06:22:15.463884 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463865    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-envoy\") pod \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") "
Oct 30 06:22:15.464153 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463911    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-socket" pod "c7295343-bb58-43a6-a460-9b26fb26edc0" (UID: "c7295343-bb58-43a6-a460-9b26fb26edc0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.464153 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.463987    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-socket\") pod \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") "
Oct 30 06:22:15.464153 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.464053    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istiod-ca-cert\") pod \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") "
Oct 30 06:22:15.464327 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.464175    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-envoy\") pod \"c7295343-bb58-43a6-a460-9b26fb26edc0\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") "
Oct 30 06:22:15.464327 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.464224    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-data\") pod \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") "
Oct 30 06:22:15.464327 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.464284    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-data\") pod \"c7295343-bb58-43a6-a460-9b26fb26edc0\" (UID: \"c7295343-bb58-43a6-a460-9b26fb26edc0\") "
Oct 30 06:22:15.464492 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.464358    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-credential-socket\") pod \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") "
Oct 30 06:22:15.464492 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.464374    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-socket" pod "8a1d2da0-194d-44b7-94ad-c49bd3cb2451" (UID: "8a1d2da0-194d-44b7-94ad-c49bd3cb2451"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.464492 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.464409    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-certs\") pod \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\" (UID: \"8a1d2da0-194d-44b7-94ad-c49bd3cb2451\") "
Oct 30 06:22:15.464711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.464561    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-certs" pod "c7295343-bb58-43a6-a460-9b26fb26edc0" (UID: "c7295343-bb58-43a6-a460-9b26fb26edc0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.464711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.464580    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7295343-bb58-43a6-a460-9b26fb26edc0-istiod-ca-cert" pod "c7295343-bb58-43a6-a460-9b26fb26edc0" (UID: "c7295343-bb58-43a6-a460-9b26fb26edc0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:15.465894 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.464940    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istiod-ca-cert" pod "8a1d2da0-194d-44b7-94ad-c49bd3cb2451" (UID: "8a1d2da0-194d-44b7-94ad-c49bd3cb2451"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:15.465894 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.465053    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-credential-socket" pod "c7295343-bb58-43a6-a460-9b26fb26edc0" (UID: "c7295343-bb58-43a6-a460-9b26fb26edc0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.465894 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.465491    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.465894 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.465511    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.465894 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.465539    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.465894 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.465553    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7295343-bb58-43a6-a460-9b26fb26edc0-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.465894 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.465562    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.465894 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.465570    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.469418 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.469394    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-podinfo" pod "c7295343-bb58-43a6-a460-9b26fb26edc0" (UID: "c7295343-bb58-43a6-a460-9b26fb26edc0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:22:15.473005 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.472982    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-data" pod "c7295343-bb58-43a6-a460-9b26fb26edc0" (UID: "c7295343-bb58-43a6-a460-9b26fb26edc0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.473211 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.473188    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-data" pod "8a1d2da0-194d-44b7-94ad-c49bd3cb2451" (UID: "8a1d2da0-194d-44b7-94ad-c49bd3cb2451"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.475061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.475039    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-certs" pod "8a1d2da0-194d-44b7-94ad-c49bd3cb2451" (UID: "8a1d2da0-194d-44b7-94ad-c49bd3cb2451"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.475427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.475402    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-credential-socket" pod "8a1d2da0-194d-44b7-94ad-c49bd3cb2451" (UID: "8a1d2da0-194d-44b7-94ad-c49bd3cb2451"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.476945 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.476919    5066 generic.go:356] "Generic (PLEG): container finished" podID="c7295343-bb58-43a6-a460-9b26fb26edc0" containerID="44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d" exitCode=0
Oct 30 06:22:15.477038 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.477018    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n" event={"ID":"c7295343-bb58-43a6-a460-9b26fb26edc0","Type":"ContainerDied","Data":"44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d"}
Oct 30 06:22:15.478469 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.478091    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"
Oct 30 06:22:15.481770 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.479564    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-token" pod "c7295343-bb58-43a6-a460-9b26fb26edc0" (UID: "c7295343-bb58-43a6-a460-9b26fb26edc0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:15.481941 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.481899    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n" event={"ID":"c7295343-bb58-43a6-a460-9b26fb26edc0","Type":"ContainerDied","Data":"5e353f848851b4410949f7b7e5eb9b625f997cfcf5b032dfb5cc040d7cf12acc"}
Oct 30 06:22:15.482056 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.481972    5066 scope.go:122] "RemoveContainer" containerID="44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d"
Oct 30 06:22:15.482987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.482831    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-envoy" pod "8a1d2da0-194d-44b7-94ad-c49bd3cb2451" (UID: "8a1d2da0-194d-44b7-94ad-c49bd3cb2451"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.482987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.482863    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-kube-api-access-xhsrx" pod "c7295343-bb58-43a6-a460-9b26fb26edc0" (UID: "c7295343-bb58-43a6-a460-9b26fb26edc0"). InnerVolumeSpecName "kube-api-access-xhsrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:15.482987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.482898    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-token" pod "8a1d2da0-194d-44b7-94ad-c49bd3cb2451" (UID: "8a1d2da0-194d-44b7-94ad-c49bd3cb2451"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:15.482987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.482907    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-podinfo" pod "8a1d2da0-194d-44b7-94ad-c49bd3cb2451" (UID: "8a1d2da0-194d-44b7-94ad-c49bd3cb2451"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:22:15.482987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.482897    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-envoy" pod "c7295343-bb58-43a6-a460-9b26fb26edc0" (UID: "c7295343-bb58-43a6-a460-9b26fb26edc0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.483313 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.483262    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-kube-api-access-kmwbk" pod "8a1d2da0-194d-44b7-94ad-c49bd3cb2451" (UID: "8a1d2da0-194d-44b7-94ad-c49bd3cb2451"). InnerVolumeSpecName "kube-api-access-kmwbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:15.484470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.484447    5066 generic.go:356] "Generic (PLEG): container finished" podID="8a1d2da0-194d-44b7-94ad-c49bd3cb2451" containerID="5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5" exitCode=0
Oct 30 06:22:15.484717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.484702    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"
Oct 30 06:22:15.484813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.484752    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc" event={"ID":"8a1d2da0-194d-44b7-94ad-c49bd3cb2451","Type":"ContainerDied","Data":"5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5"}
Oct 30 06:22:15.484813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.484797    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc" event={"ID":"8a1d2da0-194d-44b7-94ad-c49bd3cb2451","Type":"ContainerDied","Data":"3a665f7e9da7734d10e082438f015977c479579f82f59732b9a8a0088775951b"}
Oct 30 06:22:15.487353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.486996    5066 generic.go:356] "Generic (PLEG): container finished" podID="2a6d1b0d-6486-479c-a878-93cb39839013" containerID="89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7" exitCode=0
Oct 30 06:22:15.487353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.487103    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd" event={"ID":"2a6d1b0d-6486-479c-a878-93cb39839013","Type":"ContainerDied","Data":"89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7"}
Oct 30 06:22:15.487353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.487141    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd" event={"ID":"2a6d1b0d-6486-479c-a878-93cb39839013","Type":"ContainerDied","Data":"56d435b906fef2ffd0e45731f04482cfcfbfa40a9921305ccc44c24db52b2031"}
Oct 30 06:22:15.487353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.487210    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"
Oct 30 06:22:15.499476 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.499451    5066 scope.go:122] "RemoveContainer" containerID="44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d"
Oct 30 06:22:15.499909 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:15.499887    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d\": container with ID starting with 44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d not found: ID does not exist" containerID="44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d"
Oct 30 06:22:15.500051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.499916    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d"} err="failed to get container status \"44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d\": rpc error: code = NotFound desc = could not find container \"44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d\": container with ID starting with 44b96a95d936df2a506ff921edf16f4dff6e907234e2f1e335a8035fd4261d8d not found: ID does not exist"
Oct 30 06:22:15.500051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.499941    5066 scope.go:122] "RemoveContainer" containerID="5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5"
Oct 30 06:22:15.513982 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.513964    5066 scope.go:122] "RemoveContainer" containerID="5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5"
Oct 30 06:22:15.514349 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:15.514326    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5\": container with ID starting with 5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5 not found: ID does not exist" containerID="5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5"
Oct 30 06:22:15.514458 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.514359    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5"} err="failed to get container status \"5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5\": rpc error: code = NotFound desc = could not find container \"5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5\": container with ID starting with 5c6ab25a511d2e9064d8ebe5c9a0d9f7b27f2ab090aa687df75000fc9a4408c5 not found: ID does not exist"
Oct 30 06:22:15.514458 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.514383    5066 scope.go:122] "RemoveContainer" containerID="89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7"
Oct 30 06:22:15.539531 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.539503    5066 scope.go:122] "RemoveContainer" containerID="89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7"
Oct 30 06:22:15.539929 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:15.539899    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7\": container with ID starting with 89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7 not found: ID does not exist" containerID="89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7"
Oct 30 06:22:15.540049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.539941    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7"} err="failed to get container status \"89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7\": rpc error: code = NotFound desc = could not find container \"89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7\": container with ID starting with 89397f700001ecc7e3df25e5bec07c26c6694bea2737e72e6646cb99942f34e7 not found: ID does not exist"
Oct 30 06:22:15.557796 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:22:15.557770    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda6dbe1_e406_4d55_a0e2_1c532edda522.slice/crio-3b35959ea1f8017024c59e549be27a94cc2d42dd3d5e7c04ef2dd1c2c38db8a6 WatchSource:0}: Error finding container 3b35959ea1f8017024c59e549be27a94cc2d42dd3d5e7c04ef2dd1c2c38db8a6: Status 404 returned error can't find the container with id 3b35959ea1f8017024c59e549be27a94cc2d42dd3d5e7c04ef2dd1c2c38db8a6
Oct 30 06:22:15.559366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.559337    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"]
Oct 30 06:22:15.561745 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.561699    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:22:15.561831 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.561784    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:22:15.561831 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.561824    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:22:15.566099 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.566075    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-data\") pod \"2a6d1b0d-6486-479c-a878-93cb39839013\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") "
Oct 30 06:22:15.566394 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.566331    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-data" pod "2a6d1b0d-6486-479c-a878-93cb39839013" (UID: "2a6d1b0d-6486-479c-a878-93cb39839013"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.566491 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.566471    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-socket\") pod \"2a6d1b0d-6486-479c-a878-93cb39839013\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") "
Oct 30 06:22:15.566627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.566518    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-kube-api-access-pd8k9\" (UniqueName: \"kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-kube-api-access-pd8k9\") pod \"2a6d1b0d-6486-479c-a878-93cb39839013\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") "
Oct 30 06:22:15.566627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.566561    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-istio-token\" (UniqueName: \"kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-istio-token\") pod \"2a6d1b0d-6486-479c-a878-93cb39839013\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") "
Oct 30 06:22:15.566769 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.566719    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a6d1b0d-6486-479c-a878-93cb39839013-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a6d1b0d-6486-479c-a878-93cb39839013-istiod-ca-cert\") pod \"2a6d1b0d-6486-479c-a878-93cb39839013\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") "
Oct 30 06:22:15.566769 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.566763    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-credential-socket\") pod \"2a6d1b0d-6486-479c-a878-93cb39839013\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") "
Oct 30 06:22:15.566881 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.566815    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2a6d1b0d-6486-479c-a878-93cb39839013-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a6d1b0d-6486-479c-a878-93cb39839013-istio-podinfo\") pod \"2a6d1b0d-6486-479c-a878-93cb39839013\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") "
Oct 30 06:22:15.566937 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.566879    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-envoy\") pod \"2a6d1b0d-6486-479c-a878-93cb39839013\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") "
Oct 30 06:22:15.566937 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.566912    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-certs\") pod \"2a6d1b0d-6486-479c-a878-93cb39839013\" (UID: \"2a6d1b0d-6486-479c-a878-93cb39839013\") "
Oct 30 06:22:15.567539 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.567518    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.567815 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.567545    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.569478 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.568866    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-credential-socket" pod "2a6d1b0d-6486-479c-a878-93cb39839013" (UID: "2a6d1b0d-6486-479c-a878-93cb39839013"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.569478 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.569401    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-socket" pod "2a6d1b0d-6486-479c-a878-93cb39839013" (UID: "2a6d1b0d-6486-479c-a878-93cb39839013"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.569671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.569510    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-certs" pod "2a6d1b0d-6486-479c-a878-93cb39839013" (UID: "2a6d1b0d-6486-479c-a878-93cb39839013"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.570059 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.570039    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a6d1b0d-6486-479c-a878-93cb39839013-istiod-ca-cert" pod "2a6d1b0d-6486-479c-a878-93cb39839013" (UID: "2a6d1b0d-6486-479c-a878-93cb39839013"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:15.570199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.570169    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.570305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.570201    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.570305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.570215    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.570305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.570231    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhsrx\" (UniqueName: \"kubernetes.io/projected/c7295343-bb58-43a6-a460-9b26fb26edc0-kube-api-access-xhsrx\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.570305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.570245    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.570305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.570258    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmwbk\" (UniqueName: \"kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-kube-api-access-kmwbk\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.570305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.570273    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.570305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.570286    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.570305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.570301    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.570821 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.570317    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7295343-bb58-43a6-a460-9b26fb26edc0-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.570821 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.570335    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a1d2da0-194d-44b7-94ad-c49bd3cb2451-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.572775 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.572745    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-kube-api-access-pd8k9" pod "2a6d1b0d-6486-479c-a878-93cb39839013" (UID: "2a6d1b0d-6486-479c-a878-93cb39839013"). InnerVolumeSpecName "kube-api-access-pd8k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:15.578520 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.578495    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"]
Oct 30 06:22:15.582068 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.582044    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-envoy" pod "2a6d1b0d-6486-479c-a878-93cb39839013" (UID: "2a6d1b0d-6486-479c-a878-93cb39839013"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:15.582968 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.582937    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a6d1b0d-6486-479c-a878-93cb39839013-istio-podinfo" pod "2a6d1b0d-6486-479c-a878-93cb39839013" (UID: "2a6d1b0d-6486-479c-a878-93cb39839013"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:22:15.583254 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.583236    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-t6pjc"]
Oct 30 06:22:15.585394 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.585362    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-istio-token" pod "2a6d1b0d-6486-479c-a878-93cb39839013" (UID: "2a6d1b0d-6486-479c-a878-93cb39839013"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:15.671954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.671919    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a6d1b0d-6486-479c-a878-93cb39839013-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.671954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.671955    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.672199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.671973    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a6d1b0d-6486-479c-a878-93cb39839013-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.672199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.671989    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.672199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.672004    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.672199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.672018    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a6d1b0d-6486-479c-a878-93cb39839013-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.672199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.672031    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd8k9\" (UniqueName: \"kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-kube-api-access-pd8k9\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.672199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.672045    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a6d1b0d-6486-479c-a878-93cb39839013-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:15.882333 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.882295    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"]
Oct 30 06:22:15.890258 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.890228    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-49f6n"]
Oct 30 06:22:15.960435 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.960390    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"]
Oct 30 06:22:15.966050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:15.966014    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-5deb36-openshift-default-5c58465fc5-xqqxd"]
Oct 30 06:22:16.200242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:16.200211    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:16.406181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:16.406149    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a6d1b0d-6486-479c-a878-93cb39839013" path="/var/lib/kubelet/pods/2a6d1b0d-6486-479c-a878-93cb39839013/volumes"
Oct 30 06:22:16.406689 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:16.406675    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a1d2da0-194d-44b7-94ad-c49bd3cb2451" path="/var/lib/kubelet/pods/8a1d2da0-194d-44b7-94ad-c49bd3cb2451/volumes"
Oct 30 06:22:16.407119 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:16.407106    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7295343-bb58-43a6-a460-9b26fb26edc0" path="/var/lib/kubelet/pods/c7295343-bb58-43a6-a460-9b26fb26edc0/volumes"
Oct 30 06:22:16.498496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:16.498402    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2" event={"ID":"eda6dbe1-e406-4d55-a0e2-1c532edda522","Type":"ContainerStarted","Data":"7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295"}
Oct 30 06:22:16.498496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:16.498460    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2" event={"ID":"eda6dbe1-e406-4d55-a0e2-1c532edda522","Type":"ContainerStarted","Data":"3b35959ea1f8017024c59e549be27a94cc2d42dd3d5e7c04ef2dd1c2c38db8a6"}
Oct 30 06:22:16.557939 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:16.557867    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2" podStartSLOduration=2.557846534 podStartE2EDuration="2.557846534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:22:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:22:16.552745427 +0000 UTC m=+3040.298030773" watchObservedRunningTime="2026-10-30 06:22:16.557846534 +0000 UTC m=+3040.303131867"
Oct 30 06:22:16.695074 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:16.695030    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:16.868181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:16.868104    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:17.084234 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:17.067169    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:17.108646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:17.069538    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.182:15021/healthz/ready\": dial tcp 10.132.0.182:15021: connect: connection refused" start-of-body=
Oct 30 06:22:17.108646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:17.106732    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2" podUID="eda6dbe1-e406-4d55-a0e2-1c532edda522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.182:15021/healthz/ready\": dial tcp 10.132.0.182:15021: connect: connection refused"
Oct 30 06:22:18.062263 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:18.062229    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.182:15021/healthz/ready\": dial tcp 10.132.0.182:15021: connect: connection refused" start-of-body=
Oct 30 06:22:18.062682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:18.062292    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2" podUID="eda6dbe1-e406-4d55-a0e2-1c532edda522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.182:15021/healthz/ready\": dial tcp 10.132.0.182:15021: connect: connection refused"
Oct 30 06:22:18.143797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:18.143752    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:19.061995 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:19.061953    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.182:15021/healthz/ready\": dial tcp 10.132.0.182:15021: connect: connection refused" start-of-body=
Oct 30 06:22:19.062185 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:19.062009    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2" podUID="eda6dbe1-e406-4d55-a0e2-1c532edda522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.182:15021/healthz/ready\": dial tcp 10.132.0.182:15021: connect: connection refused"
Oct 30 06:22:20.062164 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.062123    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.182:15021/healthz/ready\": dial tcp 10.132.0.182:15021: connect: connection refused" start-of-body=
Oct 30 06:22:20.062622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.062210    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2" podUID="eda6dbe1-e406-4d55-a0e2-1c532edda522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.182:15021/healthz/ready\": dial tcp 10.132.0.182:15021: connect: connection refused"
Oct 30 06:22:20.085635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.085602    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:20.566530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.566493    5066 generic.go:356] "Generic (PLEG): container finished" podID="f575bda0-bead-41dc-a5e5-d633cad8e59d" containerID="c52bc9b8a2fca45f7cb8b8781d9d02e4f860e6b2d95c3aeb00831d57924f7c74" exitCode=0
Oct 30 06:22:20.566780 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.566718    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986" event={"ID":"f575bda0-bead-41dc-a5e5-d633cad8e59d","Type":"ContainerDied","Data":"c52bc9b8a2fca45f7cb8b8781d9d02e4f860e6b2d95c3aeb00831d57924f7c74"}
Oct 30 06:22:20.669278 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.669198    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:22:20.755569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.754145    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:20.810577 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.810535    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-podinfo\") pod \"f575bda0-bead-41dc-a5e5-d633cad8e59d\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") "
Oct 30 06:22:20.810815 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.810627    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-certs\") pod \"f575bda0-bead-41dc-a5e5-d633cad8e59d\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") "
Oct 30 06:22:20.810815 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.810671    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-data\") pod \"f575bda0-bead-41dc-a5e5-d633cad8e59d\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") "
Oct 30 06:22:20.810815 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.810714    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-envoy\") pod \"f575bda0-bead-41dc-a5e5-d633cad8e59d\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") "
Oct 30 06:22:20.810815 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.810753    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-socket\") pod \"f575bda0-bead-41dc-a5e5-d633cad8e59d\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") "
Oct 30 06:22:20.811033 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.810833    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f575bda0-bead-41dc-a5e5-d633cad8e59d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f575bda0-bead-41dc-a5e5-d633cad8e59d-istiod-ca-cert\") pod \"f575bda0-bead-41dc-a5e5-d633cad8e59d\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") "
Oct 30 06:22:20.811033 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.810874    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-credential-socket\") pod \"f575bda0-bead-41dc-a5e5-d633cad8e59d\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") "
Oct 30 06:22:20.811033 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.810976    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-token\" (UniqueName: \"kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-token\") pod \"f575bda0-bead-41dc-a5e5-d633cad8e59d\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") "
Oct 30 06:22:20.811184 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.811065    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-kube-api-access-bsrvs\" (UniqueName: \"kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-kube-api-access-bsrvs\") pod \"f575bda0-bead-41dc-a5e5-d633cad8e59d\" (UID: \"f575bda0-bead-41dc-a5e5-d633cad8e59d\") "
Oct 30 06:22:20.818089 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.818043    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f575bda0-bead-41dc-a5e5-d633cad8e59d-istiod-ca-cert" pod "f575bda0-bead-41dc-a5e5-d633cad8e59d" (UID: "f575bda0-bead-41dc-a5e5-d633cad8e59d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:20.818406 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.818359    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-socket" pod "f575bda0-bead-41dc-a5e5-d633cad8e59d" (UID: "f575bda0-bead-41dc-a5e5-d633cad8e59d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:20.818574 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.818552    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-credential-socket" pod "f575bda0-bead-41dc-a5e5-d633cad8e59d" (UID: "f575bda0-bead-41dc-a5e5-d633cad8e59d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:20.818710 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.818576    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-certs" pod "f575bda0-bead-41dc-a5e5-d633cad8e59d" (UID: "f575bda0-bead-41dc-a5e5-d633cad8e59d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:20.821665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.821631    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-podinfo" pod "f575bda0-bead-41dc-a5e5-d633cad8e59d" (UID: "f575bda0-bead-41dc-a5e5-d633cad8e59d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:22:20.821855 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.821835    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-data" pod "f575bda0-bead-41dc-a5e5-d633cad8e59d" (UID: "f575bda0-bead-41dc-a5e5-d633cad8e59d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:20.853623 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.844789    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-token" pod "f575bda0-bead-41dc-a5e5-d633cad8e59d" (UID: "f575bda0-bead-41dc-a5e5-d633cad8e59d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:20.853623 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.844893    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-envoy" pod "f575bda0-bead-41dc-a5e5-d633cad8e59d" (UID: "f575bda0-bead-41dc-a5e5-d633cad8e59d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:20.855819 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.855771    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-kube-api-access-bsrvs" pod "f575bda0-bead-41dc-a5e5-d633cad8e59d" (UID: "f575bda0-bead-41dc-a5e5-d633cad8e59d"). InnerVolumeSpecName "kube-api-access-bsrvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:20.914549 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.914511    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:20.914549 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.914551    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:20.914762 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.914566    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:20.914762 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.914577    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:20.914762 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.914624    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:20.914762 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.914639    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f575bda0-bead-41dc-a5e5-d633cad8e59d-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:20.914762 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.914653    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f575bda0-bead-41dc-a5e5-d633cad8e59d-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:20.914762 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.914666    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:20.914762 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.914680    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsrvs\" (UniqueName: \"kubernetes.io/projected/f575bda0-bead-41dc-a5e5-d633cad8e59d-kube-api-access-bsrvs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:20.987555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:20.987469    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:21.066687 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:21.066647    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.182:15021/healthz/ready\": dial tcp 10.132.0.182:15021: connect: connection refused" start-of-body=
Oct 30 06:22:21.073704 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:21.072492    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2" podUID="eda6dbe1-e406-4d55-a0e2-1c532edda522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.182:15021/healthz/ready\": dial tcp 10.132.0.182:15021: connect: connection refused"
Oct 30 06:22:21.144361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:21.144323    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"]
Oct 30 06:22:21.144940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:21.144889    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" containerID="cri-o://39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c" gracePeriod=30
Oct 30 06:22:21.206328 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:21.206297    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:21.575093 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:21.575062    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"
Oct 30 06:22:21.575306 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:21.575058    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986" event={"ID":"f575bda0-bead-41dc-a5e5-d633cad8e59d","Type":"ContainerDied","Data":"b440dcc5c5c96ba0d4c95c4df897864c410defdf3527a54a7d6615fa962e8b4d"}
Oct 30 06:22:21.575306 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:21.575225    5066 scope.go:122] "RemoveContainer" containerID="c52bc9b8a2fca45f7cb8b8781d9d02e4f860e6b2d95c3aeb00831d57924f7c74"
Oct 30 06:22:21.694118 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:21.693647    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"]
Oct 30 06:22:21.722615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:21.716796    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-24db22-openshift-default-879d78fb5-sr986"]
Oct 30 06:22:22.061490 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:22.061455    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.182:15021/healthz/ready\": dial tcp 10.132.0.182:15021: connect: connection refused" start-of-body=
Oct 30 06:22:22.061702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:22.061522    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2" podUID="eda6dbe1-e406-4d55-a0e2-1c532edda522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.182:15021/healthz/ready\": dial tcp 10.132.0.182:15021: connect: connection refused"
Oct 30 06:22:22.143944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:22.143900    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:22.340373 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:22.340286    5066 patch_prober.go:29] interesting pod/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:22:22.340373 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:22.340356    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:22:22.405214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:22.405181    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f575bda0-bead-41dc-a5e5-d633cad8e59d" path="/var/lib/kubelet/pods/f575bda0-bead-41dc-a5e5-d633cad8e59d/volumes"
Oct 30 06:22:23.067509 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:23.067479    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:23.067866 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:23.067849    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:23.068560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:23.068543    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:24.041621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:24.037692    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:25.860109 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:25.860078    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:26.061473 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.061438    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:26.242027 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.241987    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:26.543617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.541352    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 06:22:26.620682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.620367    5066 generic.go:356] "Generic (PLEG): container finished" podID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerID="39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c" exitCode=0
Oct 30 06:22:26.620682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.620481    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" event={"ID":"d7ae4c02-ca60-4ac0-b467-7be612e7faa1","Type":"ContainerDied","Data":"39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c"}
Oct 30 06:22:26.620682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.620532    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8" event={"ID":"d7ae4c02-ca60-4ac0-b467-7be612e7faa1","Type":"ContainerDied","Data":"9e7d9c09a845d42bb28dafcc21216fe9f37fdb95d49b6621e4953f6abbcbe3b3"}
Oct 30 06:22:26.620682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.620565    5066 scope.go:122] "RemoveContainer" containerID="39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c"
Oct 30 06:22:26.621069 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.620755    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"
Oct 30 06:22:26.633376 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.633184    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istiod-ca-cert\") pod \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") "
Oct 30 06:22:26.633376 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.633252    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-kube-api-access-rqxs5\" (UniqueName: \"kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-kube-api-access-rqxs5\") pod \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") "
Oct 30 06:22:26.633376 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.633359    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-credential-socket\") pod \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") "
Oct 30 06:22:26.633774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.633422    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-certs\") pod \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") "
Oct 30 06:22:26.633774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.633474    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-token\" (UniqueName: \"kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-token\") pod \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") "
Oct 30 06:22:26.633774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.633539    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-podinfo\") pod \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") "
Oct 30 06:22:26.633774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.633705    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-socket\") pod \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") "
Oct 30 06:22:26.633774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.633753    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-envoy\") pod \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") "
Oct 30 06:22:26.634130 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.633779    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-data\") pod \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\" (UID: \"d7ae4c02-ca60-4ac0-b467-7be612e7faa1\") "
Oct 30 06:22:26.634858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.634756    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-data" pod "d7ae4c02-ca60-4ac0-b467-7be612e7faa1" (UID: "d7ae4c02-ca60-4ac0-b467-7be612e7faa1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:26.635481 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.635446    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istiod-ca-cert" pod "d7ae4c02-ca60-4ac0-b467-7be612e7faa1" (UID: "d7ae4c02-ca60-4ac0-b467-7be612e7faa1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:26.637892 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.637872    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-podinfo" pod "d7ae4c02-ca60-4ac0-b467-7be612e7faa1" (UID: "d7ae4c02-ca60-4ac0-b467-7be612e7faa1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:22:26.638202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.638186    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-socket" pod "d7ae4c02-ca60-4ac0-b467-7be612e7faa1" (UID: "d7ae4c02-ca60-4ac0-b467-7be612e7faa1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:26.639964 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.639932    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-certs" pod "d7ae4c02-ca60-4ac0-b467-7be612e7faa1" (UID: "d7ae4c02-ca60-4ac0-b467-7be612e7faa1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:26.640175 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.640152    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-credential-socket" pod "d7ae4c02-ca60-4ac0-b467-7be612e7faa1" (UID: "d7ae4c02-ca60-4ac0-b467-7be612e7faa1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:26.649410 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.649389    5066 scope.go:122] "RemoveContainer" containerID="39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c"
Oct 30 06:22:26.650056 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.649981    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-envoy" pod "d7ae4c02-ca60-4ac0-b467-7be612e7faa1" (UID: "d7ae4c02-ca60-4ac0-b467-7be612e7faa1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:26.650233 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:26.650219    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c\": container with ID starting with 39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c not found: ID does not exist" containerID="39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c"
Oct 30 06:22:26.650333 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.650317    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c"} err="failed to get container status \"39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c\": rpc error: code = NotFound desc = could not find container \"39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c\": container with ID starting with 39fa1c60a77e0091d1a68280e4673b809c29f68d01dbe6aa7923e40079d2839c not found: ID does not exist"
Oct 30 06:22:26.663769 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.663739    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-token" pod "d7ae4c02-ca60-4ac0-b467-7be612e7faa1" (UID: "d7ae4c02-ca60-4ac0-b467-7be612e7faa1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:26.664001 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.663980    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-kube-api-access-rqxs5" pod "d7ae4c02-ca60-4ac0-b467-7be612e7faa1" (UID: "d7ae4c02-ca60-4ac0-b467-7be612e7faa1"). InnerVolumeSpecName "kube-api-access-rqxs5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:26.735711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.735619    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:26.735711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.735654    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:26.735711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.735667    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:26.735711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.735679    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:26.735711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.735691    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:26.735711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.735703    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:26.735711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.735715    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:26.736066 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.735726    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:26.736066 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.735739    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqxs5\" (UniqueName: \"kubernetes.io/projected/d7ae4c02-ca60-4ac0-b467-7be612e7faa1-kube-api-access-rqxs5\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:26.975120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.975058    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"]
Oct 30 06:22:26.987169 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:26.987087    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-197e9b-openshift-default-5784796cff-lxfj8"]
Oct 30 06:22:28.183654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:28.177461    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:28.403743 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:28.403710    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" path="/var/lib/kubelet/pods/d7ae4c02-ca60-4ac0-b467-7be612e7faa1/volumes"
Oct 30 06:22:30.144057 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:30.144022    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:31.019514 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:31.019478    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:31.190147 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:31.190093    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:31.610747 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:31.610516    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"]
Oct 30 06:22:31.610961 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:31.610802    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt" podUID="aaf0009d-4301-46f7-9427-47e17c7a47d6" containerName="maas-api" containerID="cri-o://9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411" gracePeriod=30
Oct 30 06:22:31.713364 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:31.713332    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:32.026655 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.026628    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 06:22:32.110614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.110530    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-metrics-certs\") pod \"aaf0009d-4301-46f7-9427-47e17c7a47d6\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") "
Oct 30 06:22:32.110818 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.110764    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aaf0009d-4301-46f7-9427-47e17c7a47d6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaf0009d-4301-46f7-9427-47e17c7a47d6-openshift-service-ca\") pod \"aaf0009d-4301-46f7-9427-47e17c7a47d6\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") "
Oct 30 06:22:32.111001 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.110825    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aaf0009d-4301-46f7-9427-47e17c7a47d6-kube-api-access-jxkwb\" (UniqueName: \"kubernetes.io/projected/aaf0009d-4301-46f7-9427-47e17c7a47d6-kube-api-access-jxkwb\") pod \"aaf0009d-4301-46f7-9427-47e17c7a47d6\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") "
Oct 30 06:22:32.111001 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.110873    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-maas-api-tls\") pod \"aaf0009d-4301-46f7-9427-47e17c7a47d6\" (UID: \"aaf0009d-4301-46f7-9427-47e17c7a47d6\") "
Oct 30 06:22:32.137611 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.127854    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaf0009d-4301-46f7-9427-47e17c7a47d6-openshift-service-ca" pod "aaf0009d-4301-46f7-9427-47e17c7a47d6" (UID: "aaf0009d-4301-46f7-9427-47e17c7a47d6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:32.137611 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.128999    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaf0009d-4301-46f7-9427-47e17c7a47d6-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:32.236798 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.233058    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-maas-api-tls" pod "aaf0009d-4301-46f7-9427-47e17c7a47d6" (UID: "aaf0009d-4301-46f7-9427-47e17c7a47d6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:22:32.253715 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.253623    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-metrics-certs" pod "aaf0009d-4301-46f7-9427-47e17c7a47d6" (UID: "aaf0009d-4301-46f7-9427-47e17c7a47d6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:22:32.255880 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.255844    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaf0009d-4301-46f7-9427-47e17c7a47d6-kube-api-access-jxkwb" pod "aaf0009d-4301-46f7-9427-47e17c7a47d6" (UID: "aaf0009d-4301-46f7-9427-47e17c7a47d6"). InnerVolumeSpecName "kube-api-access-jxkwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:32.333934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.333891    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxkwb\" (UniqueName: \"kubernetes.io/projected/aaf0009d-4301-46f7-9427-47e17c7a47d6-kube-api-access-jxkwb\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:32.333934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.333925    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:32.333934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.333935    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaf0009d-4301-46f7-9427-47e17c7a47d6-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:32.439035 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.439002    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:32.555790 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:32.555692    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:22:32.601636 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.601534    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:32.679049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.679021    5066 generic.go:356] "Generic (PLEG): container finished" podID="aaf0009d-4301-46f7-9427-47e17c7a47d6" containerID="9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411" exitCode=0
Oct 30 06:22:32.679261 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.679084    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"
Oct 30 06:22:32.679261 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.679110    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt" event={"ID":"aaf0009d-4301-46f7-9427-47e17c7a47d6","Type":"ContainerDied","Data":"9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411"}
Oct 30 06:22:32.679261 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.679178    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt" event={"ID":"aaf0009d-4301-46f7-9427-47e17c7a47d6","Type":"ContainerDied","Data":"86d0a6c630125fc2f0da418f563d593f8f16e733d18e2ba65043696e20f54d8e"}
Oct 30 06:22:32.679261 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.679209    5066 scope.go:122] "RemoveContainer" containerID="9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411"
Oct 30 06:22:32.692280 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.692260    5066 scope.go:122] "RemoveContainer" containerID="9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411"
Oct 30 06:22:32.692691 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:32.692657    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411\": container with ID starting with 9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411 not found: ID does not exist" containerID="9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411"
Oct 30 06:22:32.692813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.692701    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411"} err="failed to get container status \"9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411\": rpc error: code = NotFound desc = could not find container \"9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411\": container with ID starting with 9bd7a201d00edc03860fdf5a186214375b5f1a570f7d1480a762583d34a2e411 not found: ID does not exist"
Oct 30 06:22:32.747784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.747752    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"]
Oct 30 06:22:32.759295 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:32.759266    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-f22a52-7c87c958cb-4tjqt"]
Oct 30 06:22:33.105976 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:33.105943    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:33.470685 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:33.470654    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:33.640537 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:33.640504    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:34.408676 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:34.408634    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aaf0009d-4301-46f7-9427-47e17c7a47d6" path="/var/lib/kubelet/pods/aaf0009d-4301-46f7-9427-47e17c7a47d6/volumes"
Oct 30 06:22:34.570360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:34.570327    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:34.746044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:34.745933    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:36.653208 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:36.653172    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:38.045156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.045051    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"]
Oct 30 06:22:38.045694 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.045319    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2" podUID="3289c892-df28-4817-aef4-2291dc0d3a4b" containerName="maas-api" containerID="cri-o://93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568" gracePeriod=30
Oct 30 06:22:38.185167 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.185058    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:38.473372 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.473341    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:22:38.648443 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.648396    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-maas-api-tls\") pod \"3289c892-df28-4817-aef4-2291dc0d3a4b\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") "
Oct 30 06:22:38.648786 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.648766    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3289c892-df28-4817-aef4-2291dc0d3a4b-kube-api-access-p45zk\" (UniqueName: \"kubernetes.io/projected/3289c892-df28-4817-aef4-2291dc0d3a4b-kube-api-access-p45zk\") pod \"3289c892-df28-4817-aef4-2291dc0d3a4b\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") "
Oct 30 06:22:38.649053 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.649036    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-metrics-certs\") pod \"3289c892-df28-4817-aef4-2291dc0d3a4b\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") "
Oct 30 06:22:38.649210 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.649196    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3289c892-df28-4817-aef4-2291dc0d3a4b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3289c892-df28-4817-aef4-2291dc0d3a4b-openshift-service-ca\") pod \"3289c892-df28-4817-aef4-2291dc0d3a4b\" (UID: \"3289c892-df28-4817-aef4-2291dc0d3a4b\") "
Oct 30 06:22:38.650997 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.650973    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3289c892-df28-4817-aef4-2291dc0d3a4b-openshift-service-ca" pod "3289c892-df28-4817-aef4-2291dc0d3a4b" (UID: "3289c892-df28-4817-aef4-2291dc0d3a4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:38.655445 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.655409    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-metrics-certs" pod "3289c892-df28-4817-aef4-2291dc0d3a4b" (UID: "3289c892-df28-4817-aef4-2291dc0d3a4b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:22:38.657713 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.657690    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3289c892-df28-4817-aef4-2291dc0d3a4b-kube-api-access-p45zk" pod "3289c892-df28-4817-aef4-2291dc0d3a4b" (UID: "3289c892-df28-4817-aef4-2291dc0d3a4b"). InnerVolumeSpecName "kube-api-access-p45zk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:38.669851 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.666679    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-maas-api-tls" pod "3289c892-df28-4817-aef4-2291dc0d3a4b" (UID: "3289c892-df28-4817-aef4-2291dc0d3a4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:22:38.726288 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.725496    5066 generic.go:356] "Generic (PLEG): container finished" podID="3289c892-df28-4817-aef4-2291dc0d3a4b" containerID="93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568" exitCode=0
Oct 30 06:22:38.726288 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.725550    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"
Oct 30 06:22:38.727672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.727628    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2" event={"ID":"3289c892-df28-4817-aef4-2291dc0d3a4b","Type":"ContainerDied","Data":"93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568"}
Oct 30 06:22:38.727780 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.727712    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2" event={"ID":"3289c892-df28-4817-aef4-2291dc0d3a4b","Type":"ContainerDied","Data":"d09be7215b2c8ce1f836c9aad8d617ef5e96be432e2dd9bc8e5ccd6bb1745f82"}
Oct 30 06:22:38.727780 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.727748    5066 scope.go:122] "RemoveContainer" containerID="93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568"
Oct 30 06:22:38.738385 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.738361    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:38.740155 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.740126    5066 scope.go:122] "RemoveContainer" containerID="93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568"
Oct 30 06:22:38.740456 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:38.740433    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568\": container with ID starting with 93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568 not found: ID does not exist" containerID="93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568"
Oct 30 06:22:38.740559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.740469    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568"} err="failed to get container status \"93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568\": rpc error: code = NotFound desc = could not find container \"93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568\": container with ID starting with 93da5b4976aa61c5ce8215c255f35d8237bdfd84d8c8a202864eae767e3d3568 not found: ID does not exist"
Oct 30 06:22:38.750990 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.750969    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:38.751096 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.750994    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3289c892-df28-4817-aef4-2291dc0d3a4b-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:38.751096 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.751008    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3289c892-df28-4817-aef4-2291dc0d3a4b-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:38.751096 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.751021    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p45zk\" (UniqueName: \"kubernetes.io/projected/3289c892-df28-4817-aef4-2291dc0d3a4b-kube-api-access-p45zk\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:38.821736 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.821690    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"]
Oct 30 06:22:38.824911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.824885    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-0f980e-694fc6f76f-nhnt2"]
Oct 30 06:22:38.908652 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:38.908582    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:39.117348 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.117254    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:39.146617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.124792    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"]
Oct 30 06:22:39.161616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.147201    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr" podUID="4cf69466-534b-4646-99ba-f1bbb28fb92b" containerName="maas-api" containerID="cri-o://c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2" gracePeriod=30
Oct 30 06:22:39.280411 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:39.280374    5066 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cf69466_534b_4646_99ba_f1bbb28fb92b.slice/crio-conmon-c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 06:22:39.305605 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.305494    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:39.476436 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.476396    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:39.512214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.512190    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 06:22:39.630615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.624581    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-maas-api-tls\") pod \"4cf69466-534b-4646-99ba-f1bbb28fb92b\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") "
Oct 30 06:22:39.630615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.625168    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cf69466-534b-4646-99ba-f1bbb28fb92b-kube-api-access-5ng2c\" (UniqueName: \"kubernetes.io/projected/4cf69466-534b-4646-99ba-f1bbb28fb92b-kube-api-access-5ng2c\") pod \"4cf69466-534b-4646-99ba-f1bbb28fb92b\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") "
Oct 30 06:22:39.630615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.625252    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-metrics-certs\") pod \"4cf69466-534b-4646-99ba-f1bbb28fb92b\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") "
Oct 30 06:22:39.630615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.625330    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4cf69466-534b-4646-99ba-f1bbb28fb92b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cf69466-534b-4646-99ba-f1bbb28fb92b-openshift-service-ca\") pod \"4cf69466-534b-4646-99ba-f1bbb28fb92b\" (UID: \"4cf69466-534b-4646-99ba-f1bbb28fb92b\") "
Oct 30 06:22:39.630615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.628449    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cf69466-534b-4646-99ba-f1bbb28fb92b-openshift-service-ca" pod "4cf69466-534b-4646-99ba-f1bbb28fb92b" (UID: "4cf69466-534b-4646-99ba-f1bbb28fb92b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:39.630615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.629190    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cf69466-534b-4646-99ba-f1bbb28fb92b-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:39.656338 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.653615    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-maas-api-tls" pod "4cf69466-534b-4646-99ba-f1bbb28fb92b" (UID: "4cf69466-534b-4646-99ba-f1bbb28fb92b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:22:39.677011 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.676931    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:39.678515 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.678467    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cf69466-534b-4646-99ba-f1bbb28fb92b-kube-api-access-5ng2c" pod "4cf69466-534b-4646-99ba-f1bbb28fb92b" (UID: "4cf69466-534b-4646-99ba-f1bbb28fb92b"). InnerVolumeSpecName "kube-api-access-5ng2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:39.683484 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.683432    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-metrics-certs" pod "4cf69466-534b-4646-99ba-f1bbb28fb92b" (UID: "4cf69466-534b-4646-99ba-f1bbb28fb92b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:22:39.730269 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.730175    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:39.730269 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.730201    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ng2c\" (UniqueName: \"kubernetes.io/projected/4cf69466-534b-4646-99ba-f1bbb28fb92b-kube-api-access-5ng2c\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:39.730269 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.730210    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cf69466-534b-4646-99ba-f1bbb28fb92b-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:39.737684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.735870    5066 generic.go:356] "Generic (PLEG): container finished" podID="4cf69466-534b-4646-99ba-f1bbb28fb92b" containerID="c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2" exitCode=0
Oct 30 06:22:39.737684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.736071    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr" event={"ID":"4cf69466-534b-4646-99ba-f1bbb28fb92b","Type":"ContainerDied","Data":"c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2"}
Oct 30 06:22:39.737684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.736113    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr" event={"ID":"4cf69466-534b-4646-99ba-f1bbb28fb92b","Type":"ContainerDied","Data":"6f0be8c20417aaa334383a788cb7824b7734f6aade67ca4d49b21ae7313a3051"}
Oct 30 06:22:39.737684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.736134    5066 scope.go:122] "RemoveContainer" containerID="c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2"
Oct 30 06:22:39.737684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.736258    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"
Oct 30 06:22:39.772722 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.772701    5066 scope.go:122] "RemoveContainer" containerID="c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2"
Oct 30 06:22:39.773067 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:39.773022    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2\": container with ID starting with c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2 not found: ID does not exist" containerID="c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2"
Oct 30 06:22:39.773405 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.773378    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2"} err="failed to get container status \"c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2\": rpc error: code = NotFound desc = could not find container \"c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2\": container with ID starting with c4dcb059787b76069a5cde3073f5b77ecd601dea3da2c6baefdedb00776519a2 not found: ID does not exist"
Oct 30 06:22:39.818973 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.818933    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"]
Oct 30 06:22:39.823127 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.823104    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-a35835-77d7f7496d-xtsgr"]
Oct 30 06:22:39.897054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:39.897021    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:40.084389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:40.084305    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:40.249267 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:40.249236    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:40.404395 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:40.404367    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3289c892-df28-4817-aef4-2291dc0d3a4b" path="/var/lib/kubelet/pods/3289c892-df28-4817-aef4-2291dc0d3a4b/volumes"
Oct 30 06:22:40.404952 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:40.404930    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4cf69466-534b-4646-99ba-f1bbb28fb92b" path="/var/lib/kubelet/pods/4cf69466-534b-4646-99ba-f1bbb28fb92b/volumes"
Oct 30 06:22:40.414970 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:40.414942    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:40.816385 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:40.816269    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:41.200006 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:41.199975    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:41.362557 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:41.362527    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:41.529435 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:41.529344    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:41.750540 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:41.750508    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:42.260889 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:42.260857    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:42.438635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:42.435016    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:44.132866 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:44.132830    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:44.322319 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:44.322288    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:44.526146 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:44.526112    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:44.722845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:44.722809    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:44.886526 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:44.886431    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:45.064953 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:45.064923    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:45.473946 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:45.473916    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:46.926881 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:46.926756    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:47.112565 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:47.112536    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:47.184088 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:47.184002    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"]
Oct 30 06:22:47.199275 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:47.199244    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"]
Oct 30 06:22:47.210540 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:47.210512    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"]
Oct 30 06:22:47.385554 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:47.385521    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"]
Oct 30 06:22:47.385782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:47.385751    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c" podUID="9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" containerName="istio-proxy" containerID="cri-o://1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5" gracePeriod=30
Oct 30 06:22:47.394980 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:47.394952    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"]
Oct 30 06:22:47.395177 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:47.395157    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2" podUID="eda6dbe1-e406-4d55-a0e2-1c532edda522" containerName="istio-proxy" containerID="cri-o://7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295" gracePeriod=30
Oct 30 06:22:47.400255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:47.400221    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:47.572706 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:47.571897    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:47.906312 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:47.906265    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:48.073776 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:48.073736    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:48.201669 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:48.201567    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"]
Oct 30 06:22:48.201898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:48.201807    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" podUID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" containerName="istio-proxy" containerID="cri-o://121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd" gracePeriod=30
Oct 30 06:22:48.269222 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:48.269190    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:48.996122 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:48.996086    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:49.189650 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:49.188007    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:49.228394 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:49.226482    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"]
Oct 30 06:22:49.228394 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:49.226745    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" podUID="e977c71a-0612-4f50-876e-720196d43303" containerName="istio-proxy" containerID="cri-o://848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b" gracePeriod=30
Oct 30 06:22:49.352751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:49.352666    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:50.398481 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:50.398418    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:51.025263 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:51.025127    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:51.386036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:51.385952    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:22:51.386036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:51.386014    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c" podUID="9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:22:52.686100 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.686076    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:22:52.696214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696189    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istiod-ca-cert\") pod \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") "
Oct 30 06:22:52.696361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696226    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-token\" (UniqueName: \"kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-token\") pod \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") "
Oct 30 06:22:52.696361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696261    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-credential-socket\") pod \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") "
Oct 30 06:22:52.696458 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696385    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-envoy\") pod \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") "
Oct 30 06:22:52.696458 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696439    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-kube-api-access-7vchc\" (UniqueName: \"kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-kube-api-access-7vchc\") pod \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") "
Oct 30 06:22:52.696569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696460    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-certs\") pod \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") "
Oct 30 06:22:52.696569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696486    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-data\") pod \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") "
Oct 30 06:22:52.696569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696531    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-podinfo\") pod \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") "
Oct 30 06:22:52.696569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696564    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-socket\") pod \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\" (UID: \"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d\") "
Oct 30 06:22:52.696815 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696581    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-credential-socket" pod "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" (UID: "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:52.696815 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696665    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istiod-ca-cert" pod "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" (UID: "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:52.696933 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696854    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-certs" pod "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" (UID: "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:52.696933 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.696865    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-data" pod "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" (UID: "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:52.697310 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.697236    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.697310 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.697278    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-socket" pod "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" (UID: "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:52.697310 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.697298    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.697525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.697313    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.697525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.697326    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.699217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.699190    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-envoy" pod "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" (UID: "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:52.699315 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.699225    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-token" pod "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" (UID: "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:52.699315 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.699250    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-kube-api-access-7vchc" pod "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" (UID: "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d"). InnerVolumeSpecName "kube-api-access-7vchc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:52.699999 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.699982    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-podinfo" pod "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" (UID: "9fd717f7-573f-4bc8-99b6-31ccdaa6d45d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:22:52.739505 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.739476    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:52.789005 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.787104    5066 patch_prober.go:29] interesting pod/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:22:52.789005 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.787161    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" podUID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:22:52.799075 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.799029    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-kube-api-access-vjjcs\" (UniqueName: \"kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-kube-api-access-vjjcs\") pod \"eda6dbe1-e406-4d55-a0e2-1c532edda522\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") "
Oct 30 06:22:52.799255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.799116    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eda6dbe1-e406-4d55-a0e2-1c532edda522-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eda6dbe1-e406-4d55-a0e2-1c532edda522-istiod-ca-cert\") pod \"eda6dbe1-e406-4d55-a0e2-1c532edda522\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") "
Oct 30 06:22:52.799255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.799178    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-envoy\") pod \"eda6dbe1-e406-4d55-a0e2-1c532edda522\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") "
Oct 30 06:22:52.799862 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.799802    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eda6dbe1-e406-4d55-a0e2-1c532edda522-istiod-ca-cert" pod "eda6dbe1-e406-4d55-a0e2-1c532edda522" (UID: "eda6dbe1-e406-4d55-a0e2-1c532edda522"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:52.804039 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.802969    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-kube-api-access-vjjcs" pod "eda6dbe1-e406-4d55-a0e2-1c532edda522" (UID: "eda6dbe1-e406-4d55-a0e2-1c532edda522"). InnerVolumeSpecName "kube-api-access-vjjcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:52.805435 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.805409    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-token\" (UniqueName: \"kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-token\") pod \"eda6dbe1-e406-4d55-a0e2-1c532edda522\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") "
Oct 30 06:22:52.805575 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.805465    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-certs\") pod \"eda6dbe1-e406-4d55-a0e2-1c532edda522\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") "
Oct 30 06:22:52.805672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.805580    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-podinfo\") pod \"eda6dbe1-e406-4d55-a0e2-1c532edda522\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") "
Oct 30 06:22:52.805672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.805652    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-credential-socket\") pod \"eda6dbe1-e406-4d55-a0e2-1c532edda522\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") "
Oct 30 06:22:52.805784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.805765    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-socket\") pod \"eda6dbe1-e406-4d55-a0e2-1c532edda522\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") "
Oct 30 06:22:52.805863 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.805845    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-data\") pod \"eda6dbe1-e406-4d55-a0e2-1c532edda522\" (UID: \"eda6dbe1-e406-4d55-a0e2-1c532edda522\") "
Oct 30 06:22:52.806516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.806495    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eda6dbe1-e406-4d55-a0e2-1c532edda522-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.806622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.806522    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.806622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.806535    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vchc\" (UniqueName: \"kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-kube-api-access-7vchc\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.806622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.806549    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.806622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.806562    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.806622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.806576    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.806622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.806609    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjjcs\" (UniqueName: \"kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-kube-api-access-vjjcs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.806937 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.806820    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-data" pod "eda6dbe1-e406-4d55-a0e2-1c532edda522" (UID: "eda6dbe1-e406-4d55-a0e2-1c532edda522"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:52.808566 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.808536    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-certs" pod "eda6dbe1-e406-4d55-a0e2-1c532edda522" (UID: "eda6dbe1-e406-4d55-a0e2-1c532edda522"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:52.808791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.808573    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-credential-socket" pod "eda6dbe1-e406-4d55-a0e2-1c532edda522" (UID: "eda6dbe1-e406-4d55-a0e2-1c532edda522"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:52.808791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.808774    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-socket" pod "eda6dbe1-e406-4d55-a0e2-1c532edda522" (UID: "eda6dbe1-e406-4d55-a0e2-1c532edda522"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:52.809239 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.809207    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-envoy" pod "eda6dbe1-e406-4d55-a0e2-1c532edda522" (UID: "eda6dbe1-e406-4d55-a0e2-1c532edda522"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:52.810794 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.810266    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-token" pod "eda6dbe1-e406-4d55-a0e2-1c532edda522" (UID: "eda6dbe1-e406-4d55-a0e2-1c532edda522"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:52.811539 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.811518    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-podinfo" pod "eda6dbe1-e406-4d55-a0e2-1c532edda522" (UID: "eda6dbe1-e406-4d55-a0e2-1c532edda522"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:22:52.855113 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.855077    5066 generic.go:356] "Generic (PLEG): container finished" podID="eda6dbe1-e406-4d55-a0e2-1c532edda522" containerID="7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295" exitCode=0
Oct 30 06:22:52.855294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.855123    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"
Oct 30 06:22:52.855294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.855151    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2" event={"ID":"eda6dbe1-e406-4d55-a0e2-1c532edda522","Type":"ContainerDied","Data":"7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295"}
Oct 30 06:22:52.855294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.855205    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2" event={"ID":"eda6dbe1-e406-4d55-a0e2-1c532edda522","Type":"ContainerDied","Data":"3b35959ea1f8017024c59e549be27a94cc2d42dd3d5e7c04ef2dd1c2c38db8a6"}
Oct 30 06:22:52.855294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.855228    5066 scope.go:122] "RemoveContainer" containerID="7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295"
Oct 30 06:22:52.856866 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.856841    5066 generic.go:356] "Generic (PLEG): container finished" podID="9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" containerID="1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5" exitCode=0
Oct 30 06:22:52.857016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.856893    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"
Oct 30 06:22:52.857016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.856943    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c" event={"ID":"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d","Type":"ContainerDied","Data":"1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5"}
Oct 30 06:22:52.857310 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.857013    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c" event={"ID":"9fd717f7-573f-4bc8-99b6-31ccdaa6d45d","Type":"ContainerDied","Data":"08969bf3a31af920a60299a9390dbdec57ac029d70986dd8fbd2e0a4ae4d950a"}
Oct 30 06:22:52.857777 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.857755    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"]
Oct 30 06:22:52.867801 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.867750    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"]
Oct 30 06:22:52.871186 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.871161    5066 scope.go:122] "RemoveContainer" containerID="7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295"
Oct 30 06:22:52.871644 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:52.871568    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295\": container with ID starting with 7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295 not found: ID does not exist" containerID="7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295"
Oct 30 06:22:52.871737 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.871653    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295"} err="failed to get container status \"7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295\": rpc error: code = NotFound desc = could not find container \"7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295\": container with ID starting with 7d9832f30104fb845eac84f777f343603dc1c41464989a82341229c38e9e6295 not found: ID does not exist"
Oct 30 06:22:52.871737 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.871681    5066 scope.go:122] "RemoveContainer" containerID="1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5"
Oct 30 06:22:52.880959 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.880932    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:52.884692 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.884673    5066 scope.go:122] "RemoveContainer" containerID="1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5"
Oct 30 06:22:52.885031 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:52.885007    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5\": container with ID starting with 1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5 not found: ID does not exist" containerID="1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5"
Oct 30 06:22:52.885139 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.885039    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5"} err="failed to get container status \"1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5\": rpc error: code = NotFound desc = could not find container \"1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5\": container with ID starting with 1ef501df3ad90b83fce8c3314860e1d1c646ce346d0c0e684aa2a6a42517d8d5 not found: ID does not exist"
Oct 30 06:22:52.909050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.909020    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.909344 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.909327    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.909475 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.909462    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.909633 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.909621    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.910624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.910539    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.910624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.910577    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eda6dbe1-e406-4d55-a0e2-1c532edda522-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.910890 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.910631    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda6dbe1-e406-4d55-a0e2-1c532edda522-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:52.922389 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.922356    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"]
Oct 30 06:22:52.931304 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.931273    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-fgrq2"]
Oct 30 06:22:52.967759 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.967722    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"]
Oct 30 06:22:52.971931 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:52.971904    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-0f980e-openshift-default-5fb44cbb67-mn94c"]
Oct 30 06:22:53.042772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.042736    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:53.204925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.204818    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:53.571678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.571569    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:22:53.670214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.670185    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-certs\") pod \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") "
Oct 30 06:22:53.670402 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.670258    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-token\" (UniqueName: \"kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-token\") pod \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") "
Oct 30 06:22:53.670402 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.670304    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-envoy\") pod \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") "
Oct 30 06:22:53.670402 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.670364    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-podinfo\") pod \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") "
Oct 30 06:22:53.670402 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.670401    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-socket\") pod \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") "
Oct 30 06:22:53.670749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.670442    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-credential-socket\") pod \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") "
Oct 30 06:22:53.670749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.670473    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-kube-api-access-9zn9s\" (UniqueName: \"kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-kube-api-access-9zn9s\") pod \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") "
Oct 30 06:22:53.670749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.670473    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-certs" pod "53b1f7d3-1fde-456e-92f1-8f45c89036ae" (UID: "53b1f7d3-1fde-456e-92f1-8f45c89036ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:53.670749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.670517    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istiod-ca-cert\") pod \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") "
Oct 30 06:22:53.670749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.670652    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-data\") pod \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\" (UID: \"53b1f7d3-1fde-456e-92f1-8f45c89036ae\") "
Oct 30 06:22:53.671018 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.670765    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-socket" pod "53b1f7d3-1fde-456e-92f1-8f45c89036ae" (UID: "53b1f7d3-1fde-456e-92f1-8f45c89036ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:53.671018 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.670984    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istiod-ca-cert" pod "53b1f7d3-1fde-456e-92f1-8f45c89036ae" (UID: "53b1f7d3-1fde-456e-92f1-8f45c89036ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:53.671018 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.671006    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-data" pod "53b1f7d3-1fde-456e-92f1-8f45c89036ae" (UID: "53b1f7d3-1fde-456e-92f1-8f45c89036ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:53.671361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.671316    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:53.671361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.671337    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:53.671361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.671347    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:53.671361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.671355    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:53.671625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.671368    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-credential-socket" pod "53b1f7d3-1fde-456e-92f1-8f45c89036ae" (UID: "53b1f7d3-1fde-456e-92f1-8f45c89036ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:53.672923 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.672895    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-envoy" pod "53b1f7d3-1fde-456e-92f1-8f45c89036ae" (UID: "53b1f7d3-1fde-456e-92f1-8f45c89036ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:53.673420 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.673390    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-kube-api-access-9zn9s" pod "53b1f7d3-1fde-456e-92f1-8f45c89036ae" (UID: "53b1f7d3-1fde-456e-92f1-8f45c89036ae"). InnerVolumeSpecName "kube-api-access-9zn9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:53.673530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.673432    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-token" pod "53b1f7d3-1fde-456e-92f1-8f45c89036ae" (UID: "53b1f7d3-1fde-456e-92f1-8f45c89036ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:53.673530 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.673469    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-podinfo" pod "53b1f7d3-1fde-456e-92f1-8f45c89036ae" (UID: "53b1f7d3-1fde-456e-92f1-8f45c89036ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:22:53.773163 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.773072    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:53.773163 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.773102    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:53.773163 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.773124    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53b1f7d3-1fde-456e-92f1-8f45c89036ae-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:53.773163 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.773137    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53b1f7d3-1fde-456e-92f1-8f45c89036ae-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:53.773163 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.773150    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zn9s\" (UniqueName: \"kubernetes.io/projected/53b1f7d3-1fde-456e-92f1-8f45c89036ae-kube-api-access-9zn9s\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:53.866955 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.866917    5066 generic.go:356] "Generic (PLEG): container finished" podID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" containerID="121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd" exitCode=0
Oct 30 06:22:53.867109 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.866965    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"
Oct 30 06:22:53.867109 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.867003    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" event={"ID":"53b1f7d3-1fde-456e-92f1-8f45c89036ae","Type":"ContainerDied","Data":"121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd"}
Oct 30 06:22:53.867109 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.867058    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j" event={"ID":"53b1f7d3-1fde-456e-92f1-8f45c89036ae","Type":"ContainerDied","Data":"4f57f46ed627110bec0ecf6f1509970c9ef47a5e1ba3004bad90a628b4cc3e0f"}
Oct 30 06:22:53.867109 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.867079    5066 scope.go:122] "RemoveContainer" containerID="121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd"
Oct 30 06:22:53.877564 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.877549    5066 scope.go:122] "RemoveContainer" containerID="121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd"
Oct 30 06:22:53.877844 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:53.877827    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd\": container with ID starting with 121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd not found: ID does not exist" containerID="121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd"
Oct 30 06:22:53.877914 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.877852    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd"} err="failed to get container status \"121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd\": rpc error: code = NotFound desc = could not find container \"121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd\": container with ID starting with 121e99075f0b3f658f50af0aea64059c31b1aba41b2a02fe74edcf4435334ebd not found: ID does not exist"
Oct 30 06:22:53.926415 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.926341    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"]
Oct 30 06:22:53.929935 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:53.929912    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6b5b1e-openshift-default-66fc85c695-8l82j"]
Oct 30 06:22:54.269746 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:54.269715    5066 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode977c71a_0612_4f50_876e_720196d43303.slice/crio-conmon-848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 06:22:54.406784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.404280    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" path="/var/lib/kubelet/pods/53b1f7d3-1fde-456e-92f1-8f45c89036ae/volumes"
Oct 30 06:22:54.406784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.405031    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" path="/var/lib/kubelet/pods/9fd717f7-573f-4bc8-99b6-31ccdaa6d45d/volumes"
Oct 30 06:22:54.406784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.405677    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eda6dbe1-e406-4d55-a0e2-1c532edda522" path="/var/lib/kubelet/pods/eda6dbe1-e406-4d55-a0e2-1c532edda522/volumes"
Oct 30 06:22:54.518070 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.518048    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 06:22:54.668501 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.668461    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:54.684404 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.684362    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-istio-token\" (UniqueName: \"kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-istio-token\") pod \"e977c71a-0612-4f50-876e-720196d43303\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") "
Oct 30 06:22:54.685147 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.685123    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-data\") pod \"e977c71a-0612-4f50-876e-720196d43303\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") "
Oct 30 06:22:54.685366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.685341    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e977c71a-0612-4f50-876e-720196d43303-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977c71a-0612-4f50-876e-720196d43303-istio-podinfo\") pod \"e977c71a-0612-4f50-876e-720196d43303\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") "
Oct 30 06:22:54.685504 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.685429    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e977c71a-0612-4f50-876e-720196d43303-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977c71a-0612-4f50-876e-720196d43303-istiod-ca-cert\") pod \"e977c71a-0612-4f50-876e-720196d43303\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") "
Oct 30 06:22:54.685504 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.685462    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-certs\") pod \"e977c71a-0612-4f50-876e-720196d43303\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") "
Oct 30 06:22:54.685678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.685519    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-socket\") pod \"e977c71a-0612-4f50-876e-720196d43303\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") "
Oct 30 06:22:54.685678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.685561    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-kube-api-access-hn24b\" (UniqueName: \"kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-kube-api-access-hn24b\") pod \"e977c71a-0612-4f50-876e-720196d43303\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") "
Oct 30 06:22:54.685678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.685652    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-envoy\") pod \"e977c71a-0612-4f50-876e-720196d43303\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") "
Oct 30 06:22:54.685847 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.685693    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-credential-socket\") pod \"e977c71a-0612-4f50-876e-720196d43303\" (UID: \"e977c71a-0612-4f50-876e-720196d43303\") "
Oct 30 06:22:54.691580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.686576    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-credential-socket" pod "e977c71a-0612-4f50-876e-720196d43303" (UID: "e977c71a-0612-4f50-876e-720196d43303"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:54.691580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.686805    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-data" pod "e977c71a-0612-4f50-876e-720196d43303" (UID: "e977c71a-0612-4f50-876e-720196d43303"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:54.691580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.687228    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-socket" pod "e977c71a-0612-4f50-876e-720196d43303" (UID: "e977c71a-0612-4f50-876e-720196d43303"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:54.691580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.687505    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e977c71a-0612-4f50-876e-720196d43303-istiod-ca-cert" pod "e977c71a-0612-4f50-876e-720196d43303" (UID: "e977c71a-0612-4f50-876e-720196d43303"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:22:54.691580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.687766    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-certs" pod "e977c71a-0612-4f50-876e-720196d43303" (UID: "e977c71a-0612-4f50-876e-720196d43303"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:54.707403 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.707365    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-envoy" pod "e977c71a-0612-4f50-876e-720196d43303" (UID: "e977c71a-0612-4f50-876e-720196d43303"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:22:54.708020 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.707993    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-istio-token" pod "e977c71a-0612-4f50-876e-720196d43303" (UID: "e977c71a-0612-4f50-876e-720196d43303"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:54.709477 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.709451    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e977c71a-0612-4f50-876e-720196d43303-istio-podinfo" pod "e977c71a-0612-4f50-876e-720196d43303" (UID: "e977c71a-0612-4f50-876e-720196d43303"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:22:54.715815 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.715790    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-kube-api-access-hn24b" pod "e977c71a-0612-4f50-876e-720196d43303" (UID: "e977c71a-0612-4f50-876e-720196d43303"). InnerVolumeSpecName "kube-api-access-hn24b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:22:54.787535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.787449    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:54.787535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.787474    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:54.787535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.787483    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977c71a-0612-4f50-876e-720196d43303-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:54.787535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.787491    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977c71a-0612-4f50-876e-720196d43303-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:54.787535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.787505    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:54.787535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.787518    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:54.787535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.787527    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn24b\" (UniqueName: \"kubernetes.io/projected/e977c71a-0612-4f50-876e-720196d43303-kube-api-access-hn24b\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:54.787535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.787536    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:54.788220 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.787546    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977c71a-0612-4f50-876e-720196d43303-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:22:54.874653 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.874622    5066 generic.go:356] "Generic (PLEG): container finished" podID="e977c71a-0612-4f50-876e-720196d43303" containerID="848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b" exitCode=0
Oct 30 06:22:54.874653 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.874652    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"
Oct 30 06:22:54.874900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.874733    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" event={"ID":"e977c71a-0612-4f50-876e-720196d43303","Type":"ContainerDied","Data":"848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b"}
Oct 30 06:22:54.874900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.874788    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9" event={"ID":"e977c71a-0612-4f50-876e-720196d43303","Type":"ContainerDied","Data":"3dcf1947e83e66dfad00c837d265fce1d95bd3ca290a718b0d830e274ec8c334"}
Oct 30 06:22:54.874900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.874816    5066 scope.go:122] "RemoveContainer" containerID="848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b"
Oct 30 06:22:54.885289 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.885268    5066 scope.go:122] "RemoveContainer" containerID="848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b"
Oct 30 06:22:54.885568 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:22:54.885544    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b\": container with ID starting with 848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b not found: ID does not exist" containerID="848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b"
Oct 30 06:22:54.885659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.885578    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b"} err="failed to get container status \"848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b\": rpc error: code = NotFound desc = could not find container \"848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b\": container with ID starting with 848b690d289599796361228274a3c35e747e080c2cc6dd89cd274d435c26e55b not found: ID does not exist"
Oct 30 06:22:54.934923 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.934892    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"]
Oct 30 06:22:54.941961 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:54.941936    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-a35835-openshift-default-56c7cfb9d5-fk8h9"]
Oct 30 06:22:56.402475 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:56.402445    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e977c71a-0612-4f50-876e-720196d43303" path="/var/lib/kubelet/pods/e977c71a-0612-4f50-876e-720196d43303/volumes"
Oct 30 06:22:57.027294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:57.027259    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:58.745906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:58.745873    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:58.909156 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:58.909094    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:59.499919 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:59.499883    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:22:59.661148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:22:59.661102    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:00.729720 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:00.729688    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:02.208032 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.207992    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main" containerID="cri-o://adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c" gracePeriod=45
Oct 30 06:23:02.224140 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.224090    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" containerID="cri-o://e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e" gracePeriod=45
Oct 30 06:23:02.231987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.231821    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main" containerID="cri-o://ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd" gracePeriod=45
Oct 30 06:23:02.236828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.236793    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.67:8000/ready\": dial tcp 10.132.0.67:8000: connect: connection refused"
Oct 30 06:23:02.552188 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.548618    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 06:23:02.558826 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.558795    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 06:23:02.562232 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:23:02.562196    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:23:02.578932 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.578907    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 06:23:02.607954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.607805    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61de80c4-337b-4223-9a80-92781d29ff3b-tls-certs\" (UniqueName: \"kubernetes.io/secret/61de80c4-337b-4223-9a80-92781d29ff3b-tls-certs\") pod \"61de80c4-337b-4223-9a80-92781d29ff3b\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") "
Oct 30 06:23:02.607954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.607918    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-home\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-home\") pod \"61de80c4-337b-4223-9a80-92781d29ff3b\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") "
Oct 30 06:23:02.608291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.607965    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71a86815-9e7e-4c32-a865-e51a1c51c7c9-kube-api-access-bbgff\" (UniqueName: \"kubernetes.io/projected/71a86815-9e7e-4c32-a865-e51a1c51c7c9-kube-api-access-bbgff\") pod \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") "
Oct 30 06:23:02.608291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.607995    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-model-cache\") pod \"61de80c4-337b-4223-9a80-92781d29ff3b\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") "
Oct 30 06:23:02.608291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608026    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-model-cache\") pod \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") "
Oct 30 06:23:02.608291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608068    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-home\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-home\") pod \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") "
Oct 30 06:23:02.608291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608093    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-tmp-dir\") pod \"40749c08-d2c2-4a79-acb6-45b23d69f7db\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") "
Oct 30 06:23:02.608291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608120    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61de80c4-337b-4223-9a80-92781d29ff3b-kube-api-access-27gsc\" (UniqueName: \"kubernetes.io/projected/61de80c4-337b-4223-9a80-92781d29ff3b-kube-api-access-27gsc\") pod \"61de80c4-337b-4223-9a80-92781d29ff3b\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") "
Oct 30 06:23:02.608291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608163    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-model-cache\") pod \"40749c08-d2c2-4a79-acb6-45b23d69f7db\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") "
Oct 30 06:23:02.608291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608219    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-tmp-dir\") pod \"61de80c4-337b-4223-9a80-92781d29ff3b\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") "
Oct 30 06:23:02.608291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608256    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-home\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-home\") pod \"40749c08-d2c2-4a79-acb6-45b23d69f7db\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") "
Oct 30 06:23:02.608291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608280    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-dshm\") pod \"61de80c4-337b-4223-9a80-92781d29ff3b\" (UID: \"61de80c4-337b-4223-9a80-92781d29ff3b\") "
Oct 30 06:23:02.608893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608359    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-dshm\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-dshm\") pod \"40749c08-d2c2-4a79-acb6-45b23d69f7db\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") "
Oct 30 06:23:02.608893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608391    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tls-certs\" (UniqueName: \"kubernetes.io/secret/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tls-certs\") pod \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") "
Oct 30 06:23:02.608893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608444    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-dshm\") pod \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") "
Oct 30 06:23:02.608893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608494    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40749c08-d2c2-4a79-acb6-45b23d69f7db-kube-api-access-s5qh8\" (UniqueName: \"kubernetes.io/projected/40749c08-d2c2-4a79-acb6-45b23d69f7db-kube-api-access-s5qh8\") pod \"40749c08-d2c2-4a79-acb6-45b23d69f7db\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") "
Oct 30 06:23:02.608893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608548    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/40749c08-d2c2-4a79-acb6-45b23d69f7db-tls-certs\" (UniqueName: \"kubernetes.io/secret/40749c08-d2c2-4a79-acb6-45b23d69f7db-tls-certs\") pod \"40749c08-d2c2-4a79-acb6-45b23d69f7db\" (UID: \"40749c08-d2c2-4a79-acb6-45b23d69f7db\") "
Oct 30 06:23:02.608893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608619    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tmp-dir\") pod \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\" (UID: \"71a86815-9e7e-4c32-a865-e51a1c51c7c9\") "
Oct 30 06:23:02.609202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.608924    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-tmp-dir" pod "61de80c4-337b-4223-9a80-92781d29ff3b" (UID: "61de80c4-337b-4223-9a80-92781d29ff3b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:02.609202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.609152    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tmp-dir" pod "71a86815-9e7e-4c32-a865-e51a1c51c7c9" (UID: "71a86815-9e7e-4c32-a865-e51a1c51c7c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:02.611525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.609348    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-model-cache" pod "40749c08-d2c2-4a79-acb6-45b23d69f7db" (UID: "40749c08-d2c2-4a79-acb6-45b23d69f7db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:02.611525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.609411    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.611525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.609429    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.611525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.609543    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-home" pod "40749c08-d2c2-4a79-acb6-45b23d69f7db" (UID: "40749c08-d2c2-4a79-acb6-45b23d69f7db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:02.611525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.609819    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-home" pod "71a86815-9e7e-4c32-a865-e51a1c51c7c9" (UID: "71a86815-9e7e-4c32-a865-e51a1c51c7c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:02.611525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.609832    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-model-cache" pod "61de80c4-337b-4223-9a80-92781d29ff3b" (UID: "61de80c4-337b-4223-9a80-92781d29ff3b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:02.611525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.610007    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-model-cache" pod "71a86815-9e7e-4c32-a865-e51a1c51c7c9" (UID: "71a86815-9e7e-4c32-a865-e51a1c51c7c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:02.611525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.610001    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-tmp-dir" pod "40749c08-d2c2-4a79-acb6-45b23d69f7db" (UID: "40749c08-d2c2-4a79-acb6-45b23d69f7db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:02.611525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.610256    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-home" pod "61de80c4-337b-4223-9a80-92781d29ff3b" (UID: "61de80c4-337b-4223-9a80-92781d29ff3b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:02.612567 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.612540    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-dshm" pod "61de80c4-337b-4223-9a80-92781d29ff3b" (UID: "61de80c4-337b-4223-9a80-92781d29ff3b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:02.613711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.613559    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tls-certs" pod "71a86815-9e7e-4c32-a865-e51a1c51c7c9" (UID: "71a86815-9e7e-4c32-a865-e51a1c51c7c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:23:02.614823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.614514    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61de80c4-337b-4223-9a80-92781d29ff3b-tls-certs" pod "61de80c4-337b-4223-9a80-92781d29ff3b" (UID: "61de80c4-337b-4223-9a80-92781d29ff3b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:23:02.614823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.614704    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-dshm" pod "71a86815-9e7e-4c32-a865-e51a1c51c7c9" (UID: "71a86815-9e7e-4c32-a865-e51a1c51c7c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:02.614823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.614766    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61de80c4-337b-4223-9a80-92781d29ff3b-kube-api-access-27gsc" pod "61de80c4-337b-4223-9a80-92781d29ff3b" (UID: "61de80c4-337b-4223-9a80-92781d29ff3b"). InnerVolumeSpecName "kube-api-access-27gsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:23:02.614823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.614796    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a86815-9e7e-4c32-a865-e51a1c51c7c9-kube-api-access-bbgff" pod "71a86815-9e7e-4c32-a865-e51a1c51c7c9" (UID: "71a86815-9e7e-4c32-a865-e51a1c51c7c9"). InnerVolumeSpecName "kube-api-access-bbgff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:23:02.615181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.615162    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40749c08-d2c2-4a79-acb6-45b23d69f7db-tls-certs" pod "40749c08-d2c2-4a79-acb6-45b23d69f7db" (UID: "40749c08-d2c2-4a79-acb6-45b23d69f7db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:23:02.615940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.615441    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-dshm" pod "40749c08-d2c2-4a79-acb6-45b23d69f7db" (UID: "40749c08-d2c2-4a79-acb6-45b23d69f7db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:02.616186 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.616168    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40749c08-d2c2-4a79-acb6-45b23d69f7db-kube-api-access-s5qh8" pod "40749c08-d2c2-4a79-acb6-45b23d69f7db" (UID: "40749c08-d2c2-4a79-acb6-45b23d69f7db"). InnerVolumeSpecName "kube-api-access-s5qh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:23:02.675548 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.675497    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:02.711630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.711571    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61de80c4-337b-4223-9a80-92781d29ff3b-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.711860 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.711844    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.711978 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.711965    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbgff\" (UniqueName: \"kubernetes.io/projected/71a86815-9e7e-4c32-a865-e51a1c51c7c9-kube-api-access-bbgff\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712065    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712088    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712101    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712115    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712129    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27gsc\" (UniqueName: \"kubernetes.io/projected/61de80c4-337b-4223-9a80-92781d29ff3b-kube-api-access-27gsc\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712141    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712155    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712167    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61de80c4-337b-4223-9a80-92781d29ff3b-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712179    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40749c08-d2c2-4a79-acb6-45b23d69f7db-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712191    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71a86815-9e7e-4c32-a865-e51a1c51c7c9-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712203    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71a86815-9e7e-4c32-a865-e51a1c51c7c9-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712216    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5qh8\" (UniqueName: \"kubernetes.io/projected/40749c08-d2c2-4a79-acb6-45b23d69f7db-kube-api-access-s5qh8\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.712253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.712231    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40749c08-d2c2-4a79-acb6-45b23d69f7db-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:02.934032 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.933998    5066 generic.go:356] "Generic (PLEG): container finished" podID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerID="e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e" exitCode=0
Oct 30 06:23:02.934241 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.934053    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"
Oct 30 06:23:02.934241 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.934088    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" event={"ID":"40749c08-d2c2-4a79-acb6-45b23d69f7db","Type":"ContainerDied","Data":"e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e"}
Oct 30 06:23:02.934241 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.934136    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h" event={"ID":"40749c08-d2c2-4a79-acb6-45b23d69f7db","Type":"ContainerDied","Data":"71adaa4aae37c795fb415596ab7acf31317aaaeccbadf4cfa5783094995b5345"}
Oct 30 06:23:02.934241 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.934157    5066 scope.go:122] "RemoveContainer" containerID="e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e"
Oct 30 06:23:02.936114 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.936089    5066 generic.go:356] "Generic (PLEG): container finished" podID="61de80c4-337b-4223-9a80-92781d29ff3b" containerID="adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c" exitCode=0
Oct 30 06:23:02.936215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.936138    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"
Oct 30 06:23:02.936215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.936149    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" event={"ID":"61de80c4-337b-4223-9a80-92781d29ff3b","Type":"ContainerDied","Data":"adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c"}
Oct 30 06:23:02.936215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.936186    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv" event={"ID":"61de80c4-337b-4223-9a80-92781d29ff3b","Type":"ContainerDied","Data":"28c0ec6fd373bdd6e574ab3be07f97308c8172eec2f2e428b3121739df05e23d"}
Oct 30 06:23:02.937993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.937962    5066 generic.go:356] "Generic (PLEG): container finished" podID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerID="ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd" exitCode=0
Oct 30 06:23:02.938108 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.937994    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" event={"ID":"71a86815-9e7e-4c32-a865-e51a1c51c7c9","Type":"ContainerDied","Data":"ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd"}
Oct 30 06:23:02.938108 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.938006    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"
Oct 30 06:23:02.938108 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.938026    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn" event={"ID":"71a86815-9e7e-4c32-a865-e51a1c51c7c9","Type":"ContainerDied","Data":"d69c83f746414e513dc059273df766ae36621e9655b2c19411fc6ea75dbf8d73"}
Oct 30 06:23:02.946446 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.946426    5066 scope.go:122] "RemoveContainer" containerID="e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e"
Oct 30 06:23:02.946826 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:23:02.946808    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e\": container with ID starting with e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e not found: ID does not exist" containerID="e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e"
Oct 30 06:23:02.946922 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.946834    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e"} err="failed to get container status \"e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e\": rpc error: code = NotFound desc = could not find container \"e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e\": container with ID starting with e68d35661c4818de58d81c2deed99d6fc0a6e79cbeee6ffe9d8d8ed2c1e0618e not found: ID does not exist"
Oct 30 06:23:02.946922 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.946851    5066 scope.go:122] "RemoveContainer" containerID="adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c"
Oct 30 06:23:02.956053 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.956037    5066 scope.go:122] "RemoveContainer" containerID="adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c"
Oct 30 06:23:02.956314 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:23:02.956297    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c\": container with ID starting with adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c not found: ID does not exist" containerID="adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c"
Oct 30 06:23:02.956384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.956323    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c"} err="failed to get container status \"adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c\": rpc error: code = NotFound desc = could not find container \"adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c\": container with ID starting with adf550dece42fc46ec3b20a78d4389ca214dee84bd848bc110100db76d1e521c not found: ID does not exist"
Oct 30 06:23:02.956384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.956344    5066 scope.go:122] "RemoveContainer" containerID="ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd"
Oct 30 06:23:02.964917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.964896    5066 scope.go:122] "RemoveContainer" containerID="ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd"
Oct 30 06:23:02.965167 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:23:02.965149    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd\": container with ID starting with ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd not found: ID does not exist" containerID="ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd"
Oct 30 06:23:02.965215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.965174    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd"} err="failed to get container status \"ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd\": rpc error: code = NotFound desc = could not find container \"ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd\": container with ID starting with ae2a744907c29ea68fe658cd676635562babf62373ffe19625228a550847bbdd not found: ID does not exist"
Oct 30 06:23:02.991716 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.991689    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"]
Oct 30 06:23:02.995957 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:02.995934    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/distinct-w6-1354a5-kserve-5d5d479cb4-gvcbv"]
Oct 30 06:23:03.034464 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:03.034423    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"]
Oct 30 06:23:03.037015 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:03.036990    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/simulator-w6-1354a5-kserve-86d8859b-mvdrn"]
Oct 30 06:23:03.074102 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:03.074071    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"]
Oct 30 06:23:03.078179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:03.078153    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-1354a5/premium-w6-1354a5-kserve-6659874989-hw89h"]
Oct 30 06:23:04.401993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:04.401963    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" path="/var/lib/kubelet/pods/40749c08-d2c2-4a79-acb6-45b23d69f7db/volumes"
Oct 30 06:23:04.402428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:04.402414    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" path="/var/lib/kubelet/pods/61de80c4-337b-4223-9a80-92781d29ff3b/volumes"
Oct 30 06:23:04.402814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:04.402802    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" path="/var/lib/kubelet/pods/71a86815-9e7e-4c32-a865-e51a1c51c7c9/volumes"
Oct 30 06:23:04.433784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:04.430966    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:06.265518 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:06.265459    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:07.888774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:07.888726    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main" containerID="cri-o://68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b" gracePeriod=45
Oct 30 06:23:07.898097 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:07.898064    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" containerID="cri-o://539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031" gracePeriod=45
Oct 30 06:23:08.092649 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.091888    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": dial tcp 10.132.0.63:8000: connect: connection refused"
Oct 30 06:23:08.243834 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.242421    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:08.307859 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.307826    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 06:23:08.310030 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.310005    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 06:23:08.409299 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.409270    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5edbb8c6-de2c-4167-8e45-ba6c78204165-kube-api-access-84snr\" (UniqueName: \"kubernetes.io/projected/5edbb8c6-de2c-4167-8e45-ba6c78204165-kube-api-access-84snr\") pod \"5edbb8c6-de2c-4167-8e45-ba6c78204165\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") "
Oct 30 06:23:08.409501 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.409312    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5edbb8c6-de2c-4167-8e45-ba6c78204165-tls-certs\" (UniqueName: \"kubernetes.io/secret/5edbb8c6-de2c-4167-8e45-ba6c78204165-tls-certs\") pod \"5edbb8c6-de2c-4167-8e45-ba6c78204165\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") "
Oct 30 06:23:08.409501 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.409375    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-tmp-dir\") pod \"a5d6f936-0eb8-4937-b596-497662f44fbc\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") "
Oct 30 06:23:08.410308 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.410284    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-model-cache\") pod \"a5d6f936-0eb8-4937-b596-497662f44fbc\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") "
Oct 30 06:23:08.410696 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.410515    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-tmp-dir" pod "a5d6f936-0eb8-4937-b596-497662f44fbc" (UID: "a5d6f936-0eb8-4937-b596-497662f44fbc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:08.410784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.410666    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-model-cache" pod "a5d6f936-0eb8-4937-b596-497662f44fbc" (UID: "a5d6f936-0eb8-4937-b596-497662f44fbc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:08.410784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.410681    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-tmp-dir\") pod \"5edbb8c6-de2c-4167-8e45-ba6c78204165\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") "
Oct 30 06:23:08.410784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.410773    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-model-cache\") pod \"5edbb8c6-de2c-4167-8e45-ba6c78204165\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") "
Oct 30 06:23:08.410987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.410816    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-home\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-home\") pod \"5edbb8c6-de2c-4167-8e45-ba6c78204165\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") "
Oct 30 06:23:08.410987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.410905    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-dshm\") pod \"a5d6f936-0eb8-4937-b596-497662f44fbc\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") "
Oct 30 06:23:08.410987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.410956    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-dshm\") pod \"5edbb8c6-de2c-4167-8e45-ba6c78204165\" (UID: \"5edbb8c6-de2c-4167-8e45-ba6c78204165\") "
Oct 30 06:23:08.411214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.411002    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-home\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-home\") pod \"a5d6f936-0eb8-4937-b596-497662f44fbc\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") "
Oct 30 06:23:08.411214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.411049    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5d6f936-0eb8-4937-b596-497662f44fbc-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5d6f936-0eb8-4937-b596-497662f44fbc-tls-certs\") pod \"a5d6f936-0eb8-4937-b596-497662f44fbc\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") "
Oct 30 06:23:08.411214 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.411106    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5d6f936-0eb8-4937-b596-497662f44fbc-kube-api-access-xznsw\" (UniqueName: \"kubernetes.io/projected/a5d6f936-0eb8-4937-b596-497662f44fbc-kube-api-access-xznsw\") pod \"a5d6f936-0eb8-4937-b596-497662f44fbc\" (UID: \"a5d6f936-0eb8-4937-b596-497662f44fbc\") "
Oct 30 06:23:08.411652 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.411577    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-tmp-dir" pod "5edbb8c6-de2c-4167-8e45-ba6c78204165" (UID: "5edbb8c6-de2c-4167-8e45-ba6c78204165"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:08.411777 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.411709    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:08.411876 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.411862    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:08.412441 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.412191    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-home" pod "5edbb8c6-de2c-4167-8e45-ba6c78204165" (UID: "5edbb8c6-de2c-4167-8e45-ba6c78204165"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:08.412956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.412934    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-home" pod "a5d6f936-0eb8-4937-b596-497662f44fbc" (UID: "a5d6f936-0eb8-4937-b596-497662f44fbc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:08.413671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.413649    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-model-cache" pod "5edbb8c6-de2c-4167-8e45-ba6c78204165" (UID: "5edbb8c6-de2c-4167-8e45-ba6c78204165"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:08.417782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.417707    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5d6f936-0eb8-4937-b596-497662f44fbc-tls-certs" pod "a5d6f936-0eb8-4937-b596-497662f44fbc" (UID: "a5d6f936-0eb8-4937-b596-497662f44fbc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:23:08.418427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.418393    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-dshm" pod "a5d6f936-0eb8-4937-b596-497662f44fbc" (UID: "a5d6f936-0eb8-4937-b596-497662f44fbc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:08.419017 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.418987    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5d6f936-0eb8-4937-b596-497662f44fbc-kube-api-access-xznsw" pod "a5d6f936-0eb8-4937-b596-497662f44fbc" (UID: "a5d6f936-0eb8-4937-b596-497662f44fbc"). InnerVolumeSpecName "kube-api-access-xznsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:23:08.419355 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.419328    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-dshm" pod "5edbb8c6-de2c-4167-8e45-ba6c78204165" (UID: "5edbb8c6-de2c-4167-8e45-ba6c78204165"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:08.422565 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.422525    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5edbb8c6-de2c-4167-8e45-ba6c78204165-tls-certs" pod "5edbb8c6-de2c-4167-8e45-ba6c78204165" (UID: "5edbb8c6-de2c-4167-8e45-ba6c78204165"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:23:08.424709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.424671    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5edbb8c6-de2c-4167-8e45-ba6c78204165-kube-api-access-84snr" pod "5edbb8c6-de2c-4167-8e45-ba6c78204165" (UID: "5edbb8c6-de2c-4167-8e45-ba6c78204165"). InnerVolumeSpecName "kube-api-access-84snr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:23:08.513041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.512960    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84snr\" (UniqueName: \"kubernetes.io/projected/5edbb8c6-de2c-4167-8e45-ba6c78204165-kube-api-access-84snr\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:08.513041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.512997    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5edbb8c6-de2c-4167-8e45-ba6c78204165-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:08.513041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.513008    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:08.513041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.513017    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:08.513041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.513025    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:08.513041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.513033    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:08.513041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.513040    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5edbb8c6-de2c-4167-8e45-ba6c78204165-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:08.513041 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.513048    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5d6f936-0eb8-4937-b596-497662f44fbc-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:08.513424 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.513055    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5d6f936-0eb8-4937-b596-497662f44fbc-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:08.513424 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.513063    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xznsw\" (UniqueName: \"kubernetes.io/projected/a5d6f936-0eb8-4937-b596-497662f44fbc-kube-api-access-xznsw\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:08.880987 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.880906    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:08.980914 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.980880    5066 generic.go:356] "Generic (PLEG): container finished" podID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerID="539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031" exitCode=0
Oct 30 06:23:08.981455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.980920    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"
Oct 30 06:23:08.981455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.980964    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" event={"ID":"5edbb8c6-de2c-4167-8e45-ba6c78204165","Type":"ContainerDied","Data":"539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031"}
Oct 30 06:23:08.981455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.981004    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp" event={"ID":"5edbb8c6-de2c-4167-8e45-ba6c78204165","Type":"ContainerDied","Data":"646b3958104e0add76bf244d7e6dbe50ee8321a6545df6a14983c40cb9f0207c"}
Oct 30 06:23:08.981455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.981025    5066 scope.go:122] "RemoveContainer" containerID="539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031"
Oct 30 06:23:08.982731 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.982705    5066 generic.go:356] "Generic (PLEG): container finished" podID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerID="68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b" exitCode=0
Oct 30 06:23:08.982845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.982720    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"
Oct 30 06:23:08.982845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.982753    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" event={"ID":"a5d6f936-0eb8-4937-b596-497662f44fbc","Type":"ContainerDied","Data":"68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b"}
Oct 30 06:23:08.982845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.982791    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g" event={"ID":"a5d6f936-0eb8-4937-b596-497662f44fbc","Type":"ContainerDied","Data":"41382d7ac4af739f5385abfda1b95cbbc3b43cddc51ad486618124e0576e2585"}
Oct 30 06:23:08.992300 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.992283    5066 scope.go:122] "RemoveContainer" containerID="539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031"
Oct 30 06:23:08.992663 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:23:08.992636    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031\": container with ID starting with 539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031 not found: ID does not exist" containerID="539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031"
Oct 30 06:23:08.992762 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.992675    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031"} err="failed to get container status \"539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031\": rpc error: code = NotFound desc = could not find container \"539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031\": container with ID starting with 539a127ed8c210a666923ef04c181d38159713c64afc80a943e0fd2fd2e60031 not found: ID does not exist"
Oct 30 06:23:08.992762 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:08.992699    5066 scope.go:122] "RemoveContainer" containerID="68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b"
Oct 30 06:23:09.002786 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:09.002767    5066 scope.go:122] "RemoveContainer" containerID="68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b"
Oct 30 06:23:09.003041 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:23:09.003025    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b\": container with ID starting with 68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b not found: ID does not exist" containerID="68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b"
Oct 30 06:23:09.003087 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:09.003048    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b"} err="failed to get container status \"68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b\": rpc error: code = NotFound desc = could not find container \"68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b\": container with ID starting with 68483bc49b54aad66fa9276948230856da5e3a2eb77ef3974da14bad35f0506b not found: ID does not exist"
Oct 30 06:23:09.053717 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:09.053673    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:09.058213 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:09.057857    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"]
Oct 30 06:23:09.062369 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:09.062316    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-4a68a1/simulator-w1-4a68a1-kserve-754678b89c-dkwgp"]
Oct 30 06:23:09.093502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:09.093469    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"]
Oct 30 06:23:09.098275 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:09.098250    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-4a68a1/premium-w1-4a68a1-kserve-596d74cf4d-cmc9g"]
Oct 30 06:23:09.224279 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:09.224250    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:09.552241 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:09.552149    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:10.402887 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:10.402858    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" path="/var/lib/kubelet/pods/5edbb8c6-de2c-4167-8e45-ba6c78204165/volumes"
Oct 30 06:23:10.403316 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:10.403302    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" path="/var/lib/kubelet/pods/a5d6f936-0eb8-4937-b596-497662f44fbc/volumes"
Oct 30 06:23:11.005729 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:11.005693    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:11.160539 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:11.160504    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:11.326889 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:11.326800    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:12.936623 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:12.936574    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:14.827634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:14.826142    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:16.210694 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:16.210648    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:16.384570 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:16.384540    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:16.500165 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:16.500038    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"]
Oct 30 06:23:16.500340 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:16.500286    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x" podUID="ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" containerName="istio-proxy" containerID="cri-o://6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e" gracePeriod=30
Oct 30 06:23:16.580898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:16.580864    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:17.063977 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:17.063926    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:17.396480 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:17.396401    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:17.606866 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:17.606830    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:17.785845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:17.785792    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:19.342126 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:19.342080    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:21.111887 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.111852    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:21.777432 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.777368    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 06:23:21.787167 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787145    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-envoy\") pod \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") "
Oct 30 06:23:21.787301 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787216    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-socket\") pod \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") "
Oct 30 06:23:21.787349 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787330    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-credential-socket\") pod \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") "
Oct 30 06:23:21.787421 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787408    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-kube-api-access-v7gl4\" (UniqueName: \"kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-kube-api-access-v7gl4\") pod \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") "
Oct 30 06:23:21.787464 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787429    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-socket" pod "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" (UID: "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:21.787499 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787476    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-data\") pod \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") "
Oct 30 06:23:21.787534 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787507    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-certs\") pod \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") "
Oct 30 06:23:21.787603 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787541    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istiod-ca-cert\") pod \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") "
Oct 30 06:23:21.787603 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787573    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-token\" (UniqueName: \"kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-token\") pod \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") "
Oct 30 06:23:21.787727 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787623    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-podinfo\") pod \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\" (UID: \"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d\") "
Oct 30 06:23:21.787727 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787644    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-credential-socket" pod "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" (UID: "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:21.787834 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787727    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-data" pod "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" (UID: "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:21.787834 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787804    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-certs" pod "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" (UID: "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:21.788029 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.787970    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istiod-ca-cert" pod "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" (UID: "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:23:21.788215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.788191    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:21.788363 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.788258    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:21.788545 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.788392    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:21.788545 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.788406    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:21.788545 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.788421    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:21.789765 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.789740    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-kube-api-access-v7gl4" pod "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" (UID: "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d"). InnerVolumeSpecName "kube-api-access-v7gl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:23:21.789870 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.789844    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-envoy" pod "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" (UID: "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:21.789940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.789921    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-token" pod "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" (UID: "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:23:21.790152 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.790130    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-podinfo" pod "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" (UID: "ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:23:21.889885 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.889855    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:21.889885 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.889881    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7gl4\" (UniqueName: \"kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-kube-api-access-v7gl4\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:21.889885 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.889892    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:21.890119 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.889903    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:21.991861 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:21.991817    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"]
Oct 30 06:23:22.078470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:22.078376    5066 generic.go:356] "Generic (PLEG): container finished" podID="ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" containerID="6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e" exitCode=0
Oct 30 06:23:22.078470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:22.078431    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"
Oct 30 06:23:22.078753 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:22.078465    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x" event={"ID":"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d","Type":"ContainerDied","Data":"6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e"}
Oct 30 06:23:22.078753 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:22.078525    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x" event={"ID":"ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d","Type":"ContainerDied","Data":"30a2b5ceae58f679a3b33c2adacc2c65e4e266d9d717f79d2d38fd3d2e1ed298"}
Oct 30 06:23:22.078753 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:22.078551    5066 scope.go:122] "RemoveContainer" containerID="6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e"
Oct 30 06:23:22.092293 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:22.092267    5066 scope.go:122] "RemoveContainer" containerID="6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e"
Oct 30 06:23:22.092624 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:23:22.092602    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e\": container with ID starting with 6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e not found: ID does not exist" containerID="6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e"
Oct 30 06:23:22.092684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:22.092636    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e"} err="failed to get container status \"6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e\": rpc error: code = NotFound desc = could not find container \"6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e\": container with ID starting with 6164ec4b943f24fcda36a327c6dbb2ec0fb74d391abf64c0c791b5f7f8f69f6e not found: ID does not exist"
Oct 30 06:23:22.139160 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:22.139124    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"]
Oct 30 06:23:22.143334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:22.143310    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c73fcb-openshift-default-59c4764f55-7wq2x"]
Oct 30 06:23:22.165766 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:22.165734    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:22.403028 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:22.402945    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" path="/var/lib/kubelet/pods/ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d/volumes"
Oct 30 06:23:23.075512 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:23.074060    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:23.468079 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:23.468047    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:23.632560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:23.632522    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:25.014134 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:25.014098    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:25.617750 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:25.617716    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:26.456961 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:26.456929    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:26.630311 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:26.630016    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:27.508174 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:27.508142    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:28.032677 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:28.032643    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:28.215978 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:28.215937    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:28.406830 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:28.406796    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:28.575841 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:28.575806    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:28.910546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:28.910515    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:29.113546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:29.113499    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:29.380503 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:29.380427    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:29.943042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:29.943007    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"]
Oct 30 06:23:29.943555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:29.943226    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n" podUID="b1193647-74ea-437e-b1ae-9040d7440d48" containerName="maas-api" containerID="cri-o://a545248539c988d3d44da4817b3b27d0ec83e824f0133edffebdda59bfb4841e" gracePeriod=30
Oct 30 06:23:30.070861 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.070823    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:30.150795 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.150764    5066 generic.go:356] "Generic (PLEG): container finished" podID="b1193647-74ea-437e-b1ae-9040d7440d48" containerID="a545248539c988d3d44da4817b3b27d0ec83e824f0133edffebdda59bfb4841e" exitCode=0
Oct 30 06:23:30.150976 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.150840    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n" event={"ID":"b1193647-74ea-437e-b1ae-9040d7440d48","Type":"ContainerDied","Data":"a545248539c988d3d44da4817b3b27d0ec83e824f0133edffebdda59bfb4841e"}
Oct 30 06:23:30.305862 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.305830    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:30.328909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.328879    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 06:23:30.419615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.416973    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-maas-api-tls\") pod \"b1193647-74ea-437e-b1ae-9040d7440d48\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") "
Oct 30 06:23:30.419615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.417266    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1193647-74ea-437e-b1ae-9040d7440d48-kube-api-access-6chkj\" (UniqueName: \"kubernetes.io/projected/b1193647-74ea-437e-b1ae-9040d7440d48-kube-api-access-6chkj\") pod \"b1193647-74ea-437e-b1ae-9040d7440d48\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") "
Oct 30 06:23:30.419615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.417355    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-metrics-certs\") pod \"b1193647-74ea-437e-b1ae-9040d7440d48\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") "
Oct 30 06:23:30.419615 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.417409    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1193647-74ea-437e-b1ae-9040d7440d48-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1193647-74ea-437e-b1ae-9040d7440d48-openshift-service-ca\") pod \"b1193647-74ea-437e-b1ae-9040d7440d48\" (UID: \"b1193647-74ea-437e-b1ae-9040d7440d48\") "
Oct 30 06:23:30.420042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.419851    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1193647-74ea-437e-b1ae-9040d7440d48-openshift-service-ca" pod "b1193647-74ea-437e-b1ae-9040d7440d48" (UID: "b1193647-74ea-437e-b1ae-9040d7440d48"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:23:30.467232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.467140    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:30.470333 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.470279    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-maas-api-tls" pod "b1193647-74ea-437e-b1ae-9040d7440d48" (UID: "b1193647-74ea-437e-b1ae-9040d7440d48"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:23:30.471808 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.471779    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1193647-74ea-437e-b1ae-9040d7440d48-kube-api-access-6chkj" pod "b1193647-74ea-437e-b1ae-9040d7440d48" (UID: "b1193647-74ea-437e-b1ae-9040d7440d48"). InnerVolumeSpecName "kube-api-access-6chkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:23:30.472752 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.472725    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-metrics-certs" pod "b1193647-74ea-437e-b1ae-9040d7440d48" (UID: "b1193647-74ea-437e-b1ae-9040d7440d48"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:23:30.519554 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.519498    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6chkj\" (UniqueName: \"kubernetes.io/projected/b1193647-74ea-437e-b1ae-9040d7440d48-kube-api-access-6chkj\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:30.519554 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.519533    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:30.519554 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.519548    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1193647-74ea-437e-b1ae-9040d7440d48-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:30.519554 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.519561    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1193647-74ea-437e-b1ae-9040d7440d48-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:30.871814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:30.871768    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:31.031051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:31.031019    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:31.158791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:31.158763    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"
Oct 30 06:23:31.158967 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:31.158764    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n" event={"ID":"b1193647-74ea-437e-b1ae-9040d7440d48","Type":"ContainerDied","Data":"ceef159985583528c6420406de04f30d86aed6cd44aa3f75ebc654e98ad458dd"}
Oct 30 06:23:31.158967 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:31.158893    5066 scope.go:122] "RemoveContainer" containerID="a545248539c988d3d44da4817b3b27d0ec83e824f0133edffebdda59bfb4841e"
Oct 30 06:23:31.199163 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:31.199136    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:31.217104 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:31.217071    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"]
Oct 30 06:23:31.230958 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:31.230932    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-4a68a1-574f9c8cc8-bbg6n"]
Oct 30 06:23:31.541937 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:31.541854    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:32.229754 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:32.229712    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:32.401885 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:32.401845    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1193647-74ea-437e-b1ae-9040d7440d48" path="/var/lib/kubelet/pods/b1193647-74ea-437e-b1ae-9040d7440d48/volumes"
Oct 30 06:23:32.552316 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:23:32.552229    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:23:32.935098 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:32.935058    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:33.116846 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:33.116817    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:33.723775 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:33.723741    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:34.676929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:34.676899    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:34.864233 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:34.864204    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:36.215877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:36.215842    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:36.389562 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:36.389532    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:36.589659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:36.589566    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:36.923224 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:36.923184    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:37.013855 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.013813    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" containerID="cri-o://7192f1a06a3553c4364aee95a0a9eef78ed8ff48099d351d40dd583a6f556f35" gracePeriod=45
Oct 30 06:23:37.018138 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.018106    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": EOF"
Oct 30 06:23:37.201458 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.201255    5066 generic.go:356] "Generic (PLEG): container finished" podID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerID="7192f1a06a3553c4364aee95a0a9eef78ed8ff48099d351d40dd583a6f556f35" exitCode=0
Oct 30 06:23:37.201458 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.201358    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" event={"ID":"e74e2be3-9336-4f70-81bf-e7caba8d1a8d","Type":"ContainerDied","Data":"7192f1a06a3553c4364aee95a0a9eef78ed8ff48099d351d40dd583a6f556f35"}
Oct 30 06:23:37.319098 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.319074    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 06:23:37.406071 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.406033    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-dshm\") pod \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") "
Oct 30 06:23:37.406644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.406581    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-home\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-home\") pod \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") "
Oct 30 06:23:37.406882 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.406858    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tls-certs\" (UniqueName: \"kubernetes.io/secret/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tls-certs\") pod \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") "
Oct 30 06:23:37.407003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.406910    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-model-cache\") pod \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") "
Oct 30 06:23:37.407419 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.407086    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-kube-api-access-bnq4q\" (UniqueName: \"kubernetes.io/projected/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-kube-api-access-bnq4q\") pod \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") "
Oct 30 06:23:37.407419 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.407136    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-home" pod "e74e2be3-9336-4f70-81bf-e7caba8d1a8d" (UID: "e74e2be3-9336-4f70-81bf-e7caba8d1a8d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:37.407633 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.407465    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-model-cache" pod "e74e2be3-9336-4f70-81bf-e7caba8d1a8d" (UID: "e74e2be3-9336-4f70-81bf-e7caba8d1a8d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:37.407705 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.407643    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tmp-dir\") pod \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\" (UID: \"e74e2be3-9336-4f70-81bf-e7caba8d1a8d\") "
Oct 30 06:23:37.408291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.408244    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:37.408291 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.408245    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tmp-dir" pod "e74e2be3-9336-4f70-81bf-e7caba8d1a8d" (UID: "e74e2be3-9336-4f70-81bf-e7caba8d1a8d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:37.408484 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.408273    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:37.409560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.409527    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-dshm" pod "e74e2be3-9336-4f70-81bf-e7caba8d1a8d" (UID: "e74e2be3-9336-4f70-81bf-e7caba8d1a8d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:23:37.410861 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.410824    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tls-certs" pod "e74e2be3-9336-4f70-81bf-e7caba8d1a8d" (UID: "e74e2be3-9336-4f70-81bf-e7caba8d1a8d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:23:37.412202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.412176    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-kube-api-access-bnq4q" pod "e74e2be3-9336-4f70-81bf-e7caba8d1a8d" (UID: "e74e2be3-9336-4f70-81bf-e7caba8d1a8d"). InnerVolumeSpecName "kube-api-access-bnq4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:23:37.509877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.509790    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:37.509877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.509817    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:37.509877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.509832    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnq4q\" (UniqueName: \"kubernetes.io/projected/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-kube-api-access-bnq4q\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:37.509877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.509843    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e74e2be3-9336-4f70-81bf-e7caba8d1a8d-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:37.971702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:37.971673    5066 scope.go:122] "RemoveContainer" containerID="7192f1a06a3553c4364aee95a0a9eef78ed8ff48099d351d40dd583a6f556f35"
Oct 30 06:23:38.209056 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:38.209015    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl" event={"ID":"e74e2be3-9336-4f70-81bf-e7caba8d1a8d","Type":"ContainerDied","Data":"5ad1256478d3b999c8545db1e1ecf015f0daa88d4513f6a1784f28b8163a8a9f"}
Oct 30 06:23:38.209056 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:38.209033    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"
Oct 30 06:23:38.300015 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:38.299921    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"]
Oct 30 06:23:38.303719 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:38.303692    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-128436/premium-w3-128436-kserve-85bd66c7c-7njnl"]
Oct 30 06:23:38.402630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:38.402576    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" path="/var/lib/kubelet/pods/e74e2be3-9336-4f70-81bf-e7caba8d1a8d/volumes"
Oct 30 06:23:38.731715 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:38.731687    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:38.915670 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:38.915638    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:39.126559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:39.126480    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:39.346298 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:39.346266    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:39.510157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:39.510123    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:41.287698 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:41.287671    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:42.691105 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:42.691041    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:42.861623 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:42.861545    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:43.058914 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:43.058835    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:43.338877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:43.338793    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:43.738611 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:43.735185    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:43.902463 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:43.902433    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:45.299242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:45.299206    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:46.901577 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:46.901543    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:47.119155 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:47.119126    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:48.161566 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:48.161533    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:48.646266 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:48.646147    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:48.840234 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:48.840195    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:50.174536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:50.174472    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:52.016412 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:52.016375    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:53.692111 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:53.692055    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:55.093253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.093212    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"]
Oct 30 06:23:55.093780 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.093502    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76" podUID="ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9" containerName="maas-api" containerID="cri-o://f0556957b969b6992749e1ed5b9f8a037ca5254354733282c5f03d1309e47374" gracePeriod=30
Oct 30 06:23:55.271972 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.271933    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:55.353802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.353140    5066 generic.go:356] "Generic (PLEG): container finished" podID="ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9" containerID="f0556957b969b6992749e1ed5b9f8a037ca5254354733282c5f03d1309e47374" exitCode=0
Oct 30 06:23:55.353802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.353334    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76" event={"ID":"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9","Type":"ContainerDied","Data":"f0556957b969b6992749e1ed5b9f8a037ca5254354733282c5f03d1309e47374"}
Oct 30 06:23:55.374938 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.374903    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"]
Oct 30 06:23:55.375146 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.375109    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99" podUID="43685c17-86ec-4839-bb32-7251aba4e869" containerName="payload-processing" containerID="cri-o://32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c" gracePeriod=30
Oct 30 06:23:55.513150 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.513121    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:55.513994 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.513971    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:23:55.588063 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.586788    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-openshift-service-ca\") pod \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") "
Oct 30 06:23:55.588063 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.586906    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-metrics-certs\") pod \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") "
Oct 30 06:23:55.588063 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.586966    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-kube-api-access-hwgrh\" (UniqueName: \"kubernetes.io/projected/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-kube-api-access-hwgrh\") pod \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") "
Oct 30 06:23:55.588063 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.587167    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-openshift-service-ca" pod "ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9" (UID: "ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:23:55.588063 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.587254    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-maas-api-tls\") pod \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\" (UID: \"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9\") "
Oct 30 06:23:55.588063 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.587761    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:55.590091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.590058    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-metrics-certs" pod "ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9" (UID: "ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:23:55.591350 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.591322    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-maas-api-tls" pod "ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9" (UID: "ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:23:55.591454 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.591426    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-kube-api-access-hwgrh" pod "ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9" (UID: "ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9"). InnerVolumeSpecName "kube-api-access-hwgrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:23:55.650534 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.650500    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"
Oct 30 06:23:55.676897 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.676848    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:55.689875 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.689786    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwgrh\" (UniqueName: \"kubernetes.io/projected/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-kube-api-access-hwgrh\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:55.689875 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.689827    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:55.689875 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.689844    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:55.790673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.790365    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43685c17-86ec-4839-bb32-7251aba4e869-kube-api-access-n4sfz\" (UniqueName: \"kubernetes.io/projected/43685c17-86ec-4839-bb32-7251aba4e869-kube-api-access-n4sfz\") pod \"43685c17-86ec-4839-bb32-7251aba4e869\" (UID: \"43685c17-86ec-4839-bb32-7251aba4e869\") "
Oct 30 06:23:55.790673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.790572    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/43685c17-86ec-4839-bb32-7251aba4e869-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43685c17-86ec-4839-bb32-7251aba4e869-plugins-config-volume\") pod \"43685c17-86ec-4839-bb32-7251aba4e869\" (UID: \"43685c17-86ec-4839-bb32-7251aba4e869\") "
Oct 30 06:23:55.791519 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.791489    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43685c17-86ec-4839-bb32-7251aba4e869-plugins-config-volume" pod "43685c17-86ec-4839-bb32-7251aba4e869" (UID: "43685c17-86ec-4839-bb32-7251aba4e869"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:23:55.795111 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.795068    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43685c17-86ec-4839-bb32-7251aba4e869-kube-api-access-n4sfz" pod "43685c17-86ec-4839-bb32-7251aba4e869" (UID: "43685c17-86ec-4839-bb32-7251aba4e869"). InnerVolumeSpecName "kube-api-access-n4sfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:23:55.892348 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.892243    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4sfz\" (UniqueName: \"kubernetes.io/projected/43685c17-86ec-4839-bb32-7251aba4e869-kube-api-access-n4sfz\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:55.892348 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.892290    5066 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43685c17-86ec-4839-bb32-7251aba4e869-plugins-config-volume\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:55.931788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:55.930425    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:56.105506 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.105477    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:56.360302 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.360274    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"
Oct 30 06:23:56.360532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.360302    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76" event={"ID":"ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9","Type":"ContainerDied","Data":"fa49cd48098f8900753cfb9d5ccb2e5fa3a1b64ed488140f6fa9f88dcf89eb8d"}
Oct 30 06:23:56.360532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.360365    5066 scope.go:122] "RemoveContainer" containerID="f0556957b969b6992749e1ed5b9f8a037ca5254354733282c5f03d1309e47374"
Oct 30 06:23:56.361852 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.361829    5066 generic.go:356] "Generic (PLEG): container finished" podID="43685c17-86ec-4839-bb32-7251aba4e869" containerID="32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c" exitCode=0
Oct 30 06:23:56.361937 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.361864    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"
Oct 30 06:23:56.361937 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.361881    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99" event={"ID":"43685c17-86ec-4839-bb32-7251aba4e869","Type":"ContainerDied","Data":"32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c"}
Oct 30 06:23:56.362051 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.361934    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99" event={"ID":"43685c17-86ec-4839-bb32-7251aba4e869","Type":"ContainerDied","Data":"39c725a96b1bcdbda1e7fd89c9fb65ad608badc7ae9387e2481b60a3d103d361"}
Oct 30 06:23:56.371749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.371578    5066 scope.go:122] "RemoveContainer" containerID="32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c"
Oct 30 06:23:56.380560 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.380542    5066 scope.go:122] "RemoveContainer" containerID="32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c"
Oct 30 06:23:56.380894 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:23:56.380874    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c\": container with ID starting with 32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c not found: ID does not exist" containerID="32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c"
Oct 30 06:23:56.380954 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.380904    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c"} err="failed to get container status \"32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c\": rpc error: code = NotFound desc = could not find container \"32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c\": container with ID starting with 32d0a2f23e8ec1ed0690bd6fda6660cb6e676993dca7b96f8ccc7ebca6dba77c not found: ID does not exist"
Oct 30 06:23:56.433510 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.433478    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"]
Oct 30 06:23:56.446480 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.446454    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d51cd8-58d79c77d5-qrk76"]
Oct 30 06:23:56.478428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.478398    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"]
Oct 30 06:23:56.480514 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.480491    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d51cd8-6697d94d98-whg99"]
Oct 30 06:23:56.635837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:56.635746    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:57.971506 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:57.971436    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:58.401294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:58.401207    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43685c17-86ec-4839-bb32-7251aba4e869" path="/var/lib/kubelet/pods/43685c17-86ec-4839-bb32-7251aba4e869/volumes"
Oct 30 06:23:58.401651 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:58.401636    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9" path="/var/lib/kubelet/pods/ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9/volumes"
Oct 30 06:23:59.096655 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.096614    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:59.437639 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.430190    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:59.438478 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.437300    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"]
Oct 30 06:23:59.438650 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.438497    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn" podUID="b64460f3-6d61-455e-923c-aa6e46ffca11" containerName="maas-api" containerID="cri-o://c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d" gracePeriod=30
Oct 30 06:23:59.571040 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.570919    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn" podUID="b64460f3-6d61-455e-923c-aa6e46ffca11" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.59:8443/health\": dial tcp 10.132.0.59:8443: connect: connection refused"
Oct 30 06:23:59.597914 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.597880    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:59.763561 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.763485    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:23:59.793438 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.793411    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 06:23:59.867054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.867006    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b64460f3-6d61-455e-923c-aa6e46ffca11-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64460f3-6d61-455e-923c-aa6e46ffca11-openshift-service-ca\") pod \"b64460f3-6d61-455e-923c-aa6e46ffca11\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") "
Oct 30 06:23:59.867244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.867134    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-metrics-certs\") pod \"b64460f3-6d61-455e-923c-aa6e46ffca11\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") "
Oct 30 06:23:59.867244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.867183    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-maas-api-tls\") pod \"b64460f3-6d61-455e-923c-aa6e46ffca11\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") "
Oct 30 06:23:59.867244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.867228    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b64460f3-6d61-455e-923c-aa6e46ffca11-kube-api-access-6mllj\" (UniqueName: \"kubernetes.io/projected/b64460f3-6d61-455e-923c-aa6e46ffca11-kube-api-access-6mllj\") pod \"b64460f3-6d61-455e-923c-aa6e46ffca11\" (UID: \"b64460f3-6d61-455e-923c-aa6e46ffca11\") "
Oct 30 06:23:59.906206 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.905536    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b64460f3-6d61-455e-923c-aa6e46ffca11-openshift-service-ca" pod "b64460f3-6d61-455e-923c-aa6e46ffca11" (UID: "b64460f3-6d61-455e-923c-aa6e46ffca11"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:23:59.917797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.910244    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b64460f3-6d61-455e-923c-aa6e46ffca11-kube-api-access-6mllj" pod "b64460f3-6d61-455e-923c-aa6e46ffca11" (UID: "b64460f3-6d61-455e-923c-aa6e46ffca11"). InnerVolumeSpecName "kube-api-access-6mllj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:23:59.917797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.910418    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-maas-api-tls" pod "b64460f3-6d61-455e-923c-aa6e46ffca11" (UID: "b64460f3-6d61-455e-923c-aa6e46ffca11"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:23:59.917797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.910734    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-metrics-certs" pod "b64460f3-6d61-455e-923c-aa6e46ffca11" (UID: "b64460f3-6d61-455e-923c-aa6e46ffca11"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:23:59.971295 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.971260    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b64460f3-6d61-455e-923c-aa6e46ffca11-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:59.971295 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.971297    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:59.971516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.971312    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b64460f3-6d61-455e-923c-aa6e46ffca11-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:23:59.971516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:23:59.971328    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mllj\" (UniqueName: \"kubernetes.io/projected/b64460f3-6d61-455e-923c-aa6e46ffca11-kube-api-access-6mllj\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:00.073671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.073502    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:00.238846 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.238738    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:00.388387 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.388300    5066 generic.go:356] "Generic (PLEG): container finished" podID="b64460f3-6d61-455e-923c-aa6e46ffca11" containerID="c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d" exitCode=0
Oct 30 06:24:00.388387 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.388348    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"
Oct 30 06:24:00.388646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.388395    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn" event={"ID":"b64460f3-6d61-455e-923c-aa6e46ffca11","Type":"ContainerDied","Data":"c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d"}
Oct 30 06:24:00.388646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.388454    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn" event={"ID":"b64460f3-6d61-455e-923c-aa6e46ffca11","Type":"ContainerDied","Data":"27fd1554d6515d5d2f94ef408cd5be8afd04bbb77be827d35392686da977683b"}
Oct 30 06:24:00.388646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.388486    5066 scope.go:122] "RemoveContainer" containerID="c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d"
Oct 30 06:24:00.397381 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.397346    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:00.399370 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.399353    5066 scope.go:122] "RemoveContainer" containerID="c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d"
Oct 30 06:24:00.399639 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:24:00.399607    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d\": container with ID starting with c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d not found: ID does not exist" containerID="c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d"
Oct 30 06:24:00.399773 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.399645    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d"} err="failed to get container status \"c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d\": rpc error: code = NotFound desc = could not find container \"c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d\": container with ID starting with c900e9ec650fd34699c9aba8571f3e012d354f9b37086807fb3120a6f49a046d not found: ID does not exist"
Oct 30 06:24:00.466815 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.466775    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"]
Oct 30 06:24:00.467232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.467211    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-128436-767b6d9869-c9rsn"]
Oct 30 06:24:00.625791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.625753    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:00.872748 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:00.872715    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:01.087986 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:01.087951    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:02.401846 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:02.401816    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b64460f3-6d61-455e-923c-aa6e46ffca11" path="/var/lib/kubelet/pods/b64460f3-6d61-455e-923c-aa6e46ffca11/volumes"
Oct 30 06:24:02.477217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:02.477177    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:02.554552 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:24:02.554524    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:24:03.108124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:03.108077    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:03.278865 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:03.278832    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:03.396086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:03.396009    5066 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" secret="" err="secret \"e2e-ipp-b-d51cd8-openshift-default-dockercfg-gdp6w\" not found"
Oct 30 06:24:03.427427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:03.426410    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"]
Oct 30 06:24:03.427427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:03.426739    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb" podUID="f66efc60-7d24-4463-8668-6af9c0951b77" containerName="istio-proxy" containerID="cri-o://c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58" gracePeriod=30
Oct 30 06:24:03.438659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:03.438626    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"]
Oct 30 06:24:03.438858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:03.438836    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr" podUID="dd898281-6242-49c8-a18b-f8d01d36863f" containerName="istio-proxy" containerID="cri-o://d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b" gracePeriod=30
Oct 30 06:24:03.441960 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:03.441931    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"]
Oct 30 06:24:03.442165 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:03.442141    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" podUID="c98c4321-bd50-417b-ae05-8def39068949" containerName="istio-proxy" containerID="cri-o://814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0" gracePeriod=30
Oct 30 06:24:03.547557 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:03.542147    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:05.078493 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:05.078462    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:05.256103 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:05.256072    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:07.104456 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:07.104406    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:07.848081 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:07.848045    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:08.011559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.011525    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:08.255736 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.255699    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"]
Oct 30 06:24:08.256257 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.255910    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerName="istio-proxy" containerID="cri-o://6bbc1fffb8cca94145e1bee5e9a0158052c6abeb8542301d1f7744c4f57cb108" gracePeriod=30
Oct 30 06:24:08.351404 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.351335    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:08.623890 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:24:08.623421    5066 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98c4321_bd50_417b_ae05_8def39068949.slice/crio-conmon-814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98c4321_bd50_417b_ae05_8def39068949.slice/crio-814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 06:24:08.734430 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.734402    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:24:08.785908 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.785864    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:24:08.786535 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.786511    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f66efc60-7d24-4463-8668-6af9c0951b77-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f66efc60-7d24-4463-8668-6af9c0951b77-istio-podinfo\") pod \"f66efc60-7d24-4463-8668-6af9c0951b77\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") "
Oct 30 06:24:08.787536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.786692    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-socket\") pod \"f66efc60-7d24-4463-8668-6af9c0951b77\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") "
Oct 30 06:24:08.787536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.786742    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-credential-socket\") pod \"f66efc60-7d24-4463-8668-6af9c0951b77\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") "
Oct 30 06:24:08.787536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.786770    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-istio-token\" (UniqueName: \"kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-istio-token\") pod \"f66efc60-7d24-4463-8668-6af9c0951b77\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") "
Oct 30 06:24:08.787536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.786804    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-envoy\") pod \"f66efc60-7d24-4463-8668-6af9c0951b77\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") "
Oct 30 06:24:08.787536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.786834    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f66efc60-7d24-4463-8668-6af9c0951b77-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f66efc60-7d24-4463-8668-6af9c0951b77-istiod-ca-cert\") pod \"f66efc60-7d24-4463-8668-6af9c0951b77\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") "
Oct 30 06:24:08.787536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.786863    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-kube-api-access-bvlzk\" (UniqueName: \"kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-kube-api-access-bvlzk\") pod \"f66efc60-7d24-4463-8668-6af9c0951b77\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") "
Oct 30 06:24:08.787536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.786893    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-certs\") pod \"f66efc60-7d24-4463-8668-6af9c0951b77\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") "
Oct 30 06:24:08.787536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.786923    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-data\") pod \"f66efc60-7d24-4463-8668-6af9c0951b77\" (UID: \"f66efc60-7d24-4463-8668-6af9c0951b77\") "
Oct 30 06:24:08.788040 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.788014    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-data" pod "f66efc60-7d24-4463-8668-6af9c0951b77" (UID: "f66efc60-7d24-4463-8668-6af9c0951b77"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.788983 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.788940    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f66efc60-7d24-4463-8668-6af9c0951b77-istiod-ca-cert" pod "f66efc60-7d24-4463-8668-6af9c0951b77" (UID: "f66efc60-7d24-4463-8668-6af9c0951b77"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:24:08.789183 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.788987    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-certs" pod "f66efc60-7d24-4463-8668-6af9c0951b77" (UID: "f66efc60-7d24-4463-8668-6af9c0951b77"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.789362 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.789217    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-credential-socket" pod "f66efc60-7d24-4463-8668-6af9c0951b77" (UID: "f66efc60-7d24-4463-8668-6af9c0951b77"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.789633 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.789609    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-socket" pod "f66efc60-7d24-4463-8668-6af9c0951b77" (UID: "f66efc60-7d24-4463-8668-6af9c0951b77"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.791722 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.791690    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f66efc60-7d24-4463-8668-6af9c0951b77-istio-podinfo" pod "f66efc60-7d24-4463-8668-6af9c0951b77" (UID: "f66efc60-7d24-4463-8668-6af9c0951b77"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:24:08.793069 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.793044    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-istio-token" pod "f66efc60-7d24-4463-8668-6af9c0951b77" (UID: "f66efc60-7d24-4463-8668-6af9c0951b77"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:24:08.793714 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.793688    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-kube-api-access-bvlzk" pod "f66efc60-7d24-4463-8668-6af9c0951b77" (UID: "f66efc60-7d24-4463-8668-6af9c0951b77"). InnerVolumeSpecName "kube-api-access-bvlzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:24:08.794044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.794014    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-envoy" pod "f66efc60-7d24-4463-8668-6af9c0951b77" (UID: "f66efc60-7d24-4463-8668-6af9c0951b77"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.814857 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.814833    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:24:08.887791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.887762    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-istio-token\" (UniqueName: \"kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-istio-token\") pod \"c98c4321-bd50-417b-ae05-8def39068949\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") "
Oct 30 06:24:08.887791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.887797    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c98c4321-bd50-417b-ae05-8def39068949-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c98c4321-bd50-417b-ae05-8def39068949-istio-podinfo\") pod \"c98c4321-bd50-417b-ae05-8def39068949\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") "
Oct 30 06:24:08.888069 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.887825    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c98c4321-bd50-417b-ae05-8def39068949-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c98c4321-bd50-417b-ae05-8def39068949-istiod-ca-cert\") pod \"c98c4321-bd50-417b-ae05-8def39068949\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") "
Oct 30 06:24:08.888069 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.887841    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd898281-6242-49c8-a18b-f8d01d36863f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd898281-6242-49c8-a18b-f8d01d36863f-istiod-ca-cert\") pod \"dd898281-6242-49c8-a18b-f8d01d36863f\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") "
Oct 30 06:24:08.888199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888089    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-socket\") pod \"dd898281-6242-49c8-a18b-f8d01d36863f\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") "
Oct 30 06:24:08.888199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888159    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-credential-socket\") pod \"dd898281-6242-49c8-a18b-f8d01d36863f\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") "
Oct 30 06:24:08.888325 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888205    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-kube-api-access-9n7mg\" (UniqueName: \"kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-kube-api-access-9n7mg\") pod \"dd898281-6242-49c8-a18b-f8d01d36863f\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") "
Oct 30 06:24:08.888325 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888211    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd898281-6242-49c8-a18b-f8d01d36863f-istiod-ca-cert" pod "dd898281-6242-49c8-a18b-f8d01d36863f" (UID: "dd898281-6242-49c8-a18b-f8d01d36863f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:24:08.888325 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888236    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-credential-socket\") pod \"c98c4321-bd50-417b-ae05-8def39068949\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") "
Oct 30 06:24:08.888325 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888263    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c98c4321-bd50-417b-ae05-8def39068949-istiod-ca-cert" pod "c98c4321-bd50-417b-ae05-8def39068949" (UID: "c98c4321-bd50-417b-ae05-8def39068949"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:24:08.888325 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888316    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-istio-token\" (UniqueName: \"kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-istio-token\") pod \"dd898281-6242-49c8-a18b-f8d01d36863f\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") "
Oct 30 06:24:08.888627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888363    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-socket\") pod \"c98c4321-bd50-417b-ae05-8def39068949\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") "
Oct 30 06:24:08.888627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888400    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-envoy\") pod \"dd898281-6242-49c8-a18b-f8d01d36863f\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") "
Oct 30 06:24:08.888627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888450    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-envoy\") pod \"c98c4321-bd50-417b-ae05-8def39068949\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") "
Oct 30 06:24:08.888627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888513    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dd898281-6242-49c8-a18b-f8d01d36863f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd898281-6242-49c8-a18b-f8d01d36863f-istio-podinfo\") pod \"dd898281-6242-49c8-a18b-f8d01d36863f\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") "
Oct 30 06:24:08.888627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888531    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-credential-socket" pod "dd898281-6242-49c8-a18b-f8d01d36863f" (UID: "dd898281-6242-49c8-a18b-f8d01d36863f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.888627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888541    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-data\") pod \"c98c4321-bd50-417b-ae05-8def39068949\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") "
Oct 30 06:24:08.888627 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888582    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-socket" pod "dd898281-6242-49c8-a18b-f8d01d36863f" (UID: "dd898281-6242-49c8-a18b-f8d01d36863f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.889756 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888776    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-socket" pod "c98c4321-bd50-417b-ae05-8def39068949" (UID: "c98c4321-bd50-417b-ae05-8def39068949"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.889756 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888786    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-certs\") pod \"dd898281-6242-49c8-a18b-f8d01d36863f\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") "
Oct 30 06:24:08.889756 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888809    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-credential-socket" pod "c98c4321-bd50-417b-ae05-8def39068949" (UID: "c98c4321-bd50-417b-ae05-8def39068949"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.889756 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888826    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-data\") pod \"dd898281-6242-49c8-a18b-f8d01d36863f\" (UID: \"dd898281-6242-49c8-a18b-f8d01d36863f\") "
Oct 30 06:24:08.889756 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.888864    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-certs\") pod \"c98c4321-bd50-417b-ae05-8def39068949\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") "
Oct 30 06:24:08.889756 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889077    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-kube-api-access-xxp5c\" (UniqueName: \"kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-kube-api-access-xxp5c\") pod \"c98c4321-bd50-417b-ae05-8def39068949\" (UID: \"c98c4321-bd50-417b-ae05-8def39068949\") "
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889760    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f66efc60-7d24-4463-8668-6af9c0951b77-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889783    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889798    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c98c4321-bd50-417b-ae05-8def39068949-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889812    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd898281-6242-49c8-a18b-f8d01d36863f-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889826    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889838    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889851    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889862    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889875    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f66efc60-7d24-4463-8668-6af9c0951b77-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889890    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvlzk\" (UniqueName: \"kubernetes.io/projected/f66efc60-7d24-4463-8668-6af9c0951b77-kube-api-access-bvlzk\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889902    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889918    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f66efc60-7d24-4463-8668-6af9c0951b77-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889931    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889972    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.890058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.889987    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.892333 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.891002    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c98c4321-bd50-417b-ae05-8def39068949-istio-podinfo" pod "c98c4321-bd50-417b-ae05-8def39068949" (UID: "c98c4321-bd50-417b-ae05-8def39068949"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:24:08.892333 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.891228    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-data" pod "dd898281-6242-49c8-a18b-f8d01d36863f" (UID: "dd898281-6242-49c8-a18b-f8d01d36863f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.892333 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.891400    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-certs" pod "dd898281-6242-49c8-a18b-f8d01d36863f" (UID: "dd898281-6242-49c8-a18b-f8d01d36863f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.892333 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.891637    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-certs" pod "c98c4321-bd50-417b-ae05-8def39068949" (UID: "c98c4321-bd50-417b-ae05-8def39068949"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.892333 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.891970    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-istio-token" pod "dd898281-6242-49c8-a18b-f8d01d36863f" (UID: "dd898281-6242-49c8-a18b-f8d01d36863f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:24:08.893817 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.892735    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-kube-api-access-xxp5c" pod "c98c4321-bd50-417b-ae05-8def39068949" (UID: "c98c4321-bd50-417b-ae05-8def39068949"). InnerVolumeSpecName "kube-api-access-xxp5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:24:08.893817 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.893074    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-kube-api-access-9n7mg" pod "dd898281-6242-49c8-a18b-f8d01d36863f" (UID: "dd898281-6242-49c8-a18b-f8d01d36863f"). InnerVolumeSpecName "kube-api-access-9n7mg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:24:08.894445 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.894410    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-data" pod "c98c4321-bd50-417b-ae05-8def39068949" (UID: "c98c4321-bd50-417b-ae05-8def39068949"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.894869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.894844    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-envoy" pod "c98c4321-bd50-417b-ae05-8def39068949" (UID: "c98c4321-bd50-417b-ae05-8def39068949"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.895014 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.894980    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-envoy" pod "dd898281-6242-49c8-a18b-f8d01d36863f" (UID: "dd898281-6242-49c8-a18b-f8d01d36863f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:08.895219 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.895198    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd898281-6242-49c8-a18b-f8d01d36863f-istio-podinfo" pod "dd898281-6242-49c8-a18b-f8d01d36863f" (UID: "dd898281-6242-49c8-a18b-f8d01d36863f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:24:08.899757 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.899551    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-istio-token" pod "c98c4321-bd50-417b-ae05-8def39068949" (UID: "c98c4321-bd50-417b-ae05-8def39068949"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:24:08.960791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.960760    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:08.991363 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.991326    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n7mg\" (UniqueName: \"kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-kube-api-access-9n7mg\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.991363 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.991358    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd898281-6242-49c8-a18b-f8d01d36863f-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.991363 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.991367    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.991648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.991375    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.991648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.991384    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd898281-6242-49c8-a18b-f8d01d36863f-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.991648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.991391    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.991648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.991399    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.991648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.991406    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd898281-6242-49c8-a18b-f8d01d36863f-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.991648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.991414    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c98c4321-bd50-417b-ae05-8def39068949-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.991648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.991421    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxp5c\" (UniqueName: \"kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-kube-api-access-xxp5c\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.991648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.991430    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c98c4321-bd50-417b-ae05-8def39068949-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:08.991648 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:08.991439    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c98c4321-bd50-417b-ae05-8def39068949-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:09.464191 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.464157    5066 generic.go:356] "Generic (PLEG): container finished" podID="f66efc60-7d24-4463-8668-6af9c0951b77" containerID="c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58" exitCode=0
Oct 30 06:24:09.464661 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.464200    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"
Oct 30 06:24:09.464661 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.464252    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb" event={"ID":"f66efc60-7d24-4463-8668-6af9c0951b77","Type":"ContainerDied","Data":"c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58"}
Oct 30 06:24:09.464661 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.464312    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb" event={"ID":"f66efc60-7d24-4463-8668-6af9c0951b77","Type":"ContainerDied","Data":"c1230cd18feda48d3a40b206089c848f5afb750cb3202cd55aafd2e8527bc1cb"}
Oct 30 06:24:09.464661 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.464344    5066 scope.go:122] "RemoveContainer" containerID="c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58"
Oct 30 06:24:09.465912 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.465890    5066 generic.go:356] "Generic (PLEG): container finished" podID="c98c4321-bd50-417b-ae05-8def39068949" containerID="814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0" exitCode=0
Oct 30 06:24:09.466016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.465940    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"
Oct 30 06:24:09.466080 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.466007    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" event={"ID":"c98c4321-bd50-417b-ae05-8def39068949","Type":"ContainerDied","Data":"814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0"}
Oct 30 06:24:09.466080 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.466071    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n" event={"ID":"c98c4321-bd50-417b-ae05-8def39068949","Type":"ContainerDied","Data":"7934ade880224dcca73c9cc73122691fc2146fb320fabb5704863c61ec291e32"}
Oct 30 06:24:09.468524 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.468504    5066 generic.go:356] "Generic (PLEG): container finished" podID="dd898281-6242-49c8-a18b-f8d01d36863f" containerID="d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b" exitCode=0
Oct 30 06:24:09.468673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.468561    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"
Oct 30 06:24:09.468673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.468576    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr" event={"ID":"dd898281-6242-49c8-a18b-f8d01d36863f","Type":"ContainerDied","Data":"d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b"}
Oct 30 06:24:09.468673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.468629    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr" event={"ID":"dd898281-6242-49c8-a18b-f8d01d36863f","Type":"ContainerDied","Data":"789282f7abb8b18ad82e38963eaa88a8c14466cd47189239de3a445644951e1c"}
Oct 30 06:24:09.476472 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.476449    5066 scope.go:122] "RemoveContainer" containerID="c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58"
Oct 30 06:24:09.476969 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:24:09.476934    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58\": container with ID starting with c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58 not found: ID does not exist" containerID="c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58"
Oct 30 06:24:09.477079 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.476970    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58"} err="failed to get container status \"c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58\": rpc error: code = NotFound desc = could not find container \"c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58\": container with ID starting with c36c2a3c471f743e2cc475c70323c15d34408890938d32e6db10877b3a501d58 not found: ID does not exist"
Oct 30 06:24:09.477079 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.476993    5066 scope.go:122] "RemoveContainer" containerID="814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0"
Oct 30 06:24:09.487181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.487161    5066 scope.go:122] "RemoveContainer" containerID="814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0"
Oct 30 06:24:09.487462 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:24:09.487444    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0\": container with ID starting with 814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0 not found: ID does not exist" containerID="814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0"
Oct 30 06:24:09.487507 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.487471    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0"} err="failed to get container status \"814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0\": rpc error: code = NotFound desc = could not find container \"814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0\": container with ID starting with 814a77f303fe716d63d8cc761dcd8a5b494e584bf58ca6f5fb7f1d8b90143fc0 not found: ID does not exist"
Oct 30 06:24:09.487507 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.487487    5066 scope.go:122] "RemoveContainer" containerID="d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b"
Oct 30 06:24:09.497056 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.497035    5066 scope.go:122] "RemoveContainer" containerID="d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b"
Oct 30 06:24:09.497394 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:24:09.497360    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b\": container with ID starting with d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b not found: ID does not exist" containerID="d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b"
Oct 30 06:24:09.497508 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.497393    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b"} err="failed to get container status \"d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b\": rpc error: code = NotFound desc = could not find container \"d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b\": container with ID starting with d8a4524373d426ebace18f8472aebba7d7707e4d96c85c3244a5acec8db5307b not found: ID does not exist"
Oct 30 06:24:09.527718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.527676    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"]
Oct 30 06:24:09.541786 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.541759    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-vr7zr"]
Oct 30 06:24:09.571117 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.571080    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"]
Oct 30 06:24:09.576678 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.576651    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9l7mb"]
Oct 30 06:24:09.613697 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.613658    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"]
Oct 30 06:24:09.617883 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:09.617853    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-d51cd8-openshift-default-d6bcdb84c-9fn6n"]
Oct 30 06:24:10.402514 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:10.402485    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c98c4321-bd50-417b-ae05-8def39068949" path="/var/lib/kubelet/pods/c98c4321-bd50-417b-ae05-8def39068949/volumes"
Oct 30 06:24:10.402974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:10.402958    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd898281-6242-49c8-a18b-f8d01d36863f" path="/var/lib/kubelet/pods/dd898281-6242-49c8-a18b-f8d01d36863f/volumes"
Oct 30 06:24:10.403363 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:10.403351    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f66efc60-7d24-4463-8668-6af9c0951b77" path="/var/lib/kubelet/pods/f66efc60-7d24-4463-8668-6af9c0951b77/volumes"
Oct 30 06:24:10.697625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:10.695804    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:12.577569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:12.577528    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:13.498361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.498319    5066 generic.go:356] "Generic (PLEG): container finished" podID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerID="6bbc1fffb8cca94145e1bee5e9a0158052c6abeb8542301d1f7744c4f57cb108" exitCode=0
Oct 30 06:24:13.498506 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.498368    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" event={"ID":"d92910fe-e3b0-41f6-92f3-aee750b56067","Type":"ContainerDied","Data":"6bbc1fffb8cca94145e1bee5e9a0158052c6abeb8542301d1f7744c4f57cb108"}
Oct 30 06:24:13.531030 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.530965    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 06:24:13.644432 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.644404    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-podinfo\") pod \"d92910fe-e3b0-41f6-92f3-aee750b56067\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") "
Oct 30 06:24:13.644929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.644450    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d92910fe-e3b0-41f6-92f3-aee750b56067-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d92910fe-e3b0-41f6-92f3-aee750b56067-istiod-ca-cert\") pod \"d92910fe-e3b0-41f6-92f3-aee750b56067\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") "
Oct 30 06:24:13.644929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.644482    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-token\" (UniqueName: \"kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-token\") pod \"d92910fe-e3b0-41f6-92f3-aee750b56067\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") "
Oct 30 06:24:13.644929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.644533    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-certs\") pod \"d92910fe-e3b0-41f6-92f3-aee750b56067\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") "
Oct 30 06:24:13.644929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.644558    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-credential-socket\") pod \"d92910fe-e3b0-41f6-92f3-aee750b56067\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") "
Oct 30 06:24:13.644929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.644641    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-socket\") pod \"d92910fe-e3b0-41f6-92f3-aee750b56067\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") "
Oct 30 06:24:13.644929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.644671    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-kube-api-access-j4df6\" (UniqueName: \"kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-kube-api-access-j4df6\") pod \"d92910fe-e3b0-41f6-92f3-aee750b56067\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") "
Oct 30 06:24:13.644929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.644696    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-envoy\") pod \"d92910fe-e3b0-41f6-92f3-aee750b56067\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") "
Oct 30 06:24:13.644929 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.644761    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-data\") pod \"d92910fe-e3b0-41f6-92f3-aee750b56067\" (UID: \"d92910fe-e3b0-41f6-92f3-aee750b56067\") "
Oct 30 06:24:13.645368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.644966    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-credential-socket" pod "d92910fe-e3b0-41f6-92f3-aee750b56067" (UID: "d92910fe-e3b0-41f6-92f3-aee750b56067"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:13.645368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.645002    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d92910fe-e3b0-41f6-92f3-aee750b56067-istiod-ca-cert" pod "d92910fe-e3b0-41f6-92f3-aee750b56067" (UID: "d92910fe-e3b0-41f6-92f3-aee750b56067"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:24:13.645368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.645182    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-certs" pod "d92910fe-e3b0-41f6-92f3-aee750b56067" (UID: "d92910fe-e3b0-41f6-92f3-aee750b56067"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:13.645368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.645091    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-socket" pod "d92910fe-e3b0-41f6-92f3-aee750b56067" (UID: "d92910fe-e3b0-41f6-92f3-aee750b56067"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:13.645368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.645199    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-data" pod "d92910fe-e3b0-41f6-92f3-aee750b56067" (UID: "d92910fe-e3b0-41f6-92f3-aee750b56067"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:13.645690 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.645672    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:13.645789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.645692    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d92910fe-e3b0-41f6-92f3-aee750b56067-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:13.645789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.645701    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:13.645789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.645708    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:13.645789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.645721    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:13.646937 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.646911    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-podinfo" pod "d92910fe-e3b0-41f6-92f3-aee750b56067" (UID: "d92910fe-e3b0-41f6-92f3-aee750b56067"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:24:13.647468 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.647447    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-token" pod "d92910fe-e3b0-41f6-92f3-aee750b56067" (UID: "d92910fe-e3b0-41f6-92f3-aee750b56067"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:24:13.647581 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.647502    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-envoy" pod "d92910fe-e3b0-41f6-92f3-aee750b56067" (UID: "d92910fe-e3b0-41f6-92f3-aee750b56067"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:24:13.647656 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.647625    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-kube-api-access-j4df6" pod "d92910fe-e3b0-41f6-92f3-aee750b56067" (UID: "d92910fe-e3b0-41f6-92f3-aee750b56067"). InnerVolumeSpecName "kube-api-access-j4df6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:24:13.746882 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.746850    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:13.746882 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.746879    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:13.746882 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.746890    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4df6\" (UniqueName: \"kubernetes.io/projected/d92910fe-e3b0-41f6-92f3-aee750b56067-kube-api-access-j4df6\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:13.747131 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:13.746899    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d92910fe-e3b0-41f6-92f3-aee750b56067-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:14.285021 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:14.284989    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:14.507302 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:14.507260    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks" event={"ID":"d92910fe-e3b0-41f6-92f3-aee750b56067","Type":"ContainerDied","Data":"29c1ce74af424b40d72cb130b4cb93a282f4b846319eb0473b89133e0d0da245"}
Oct 30 06:24:14.507302 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:14.507299    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"
Oct 30 06:24:14.507564 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:14.507347    5066 scope.go:122] "RemoveContainer" containerID="6bbc1fffb8cca94145e1bee5e9a0158052c6abeb8542301d1f7744c4f57cb108"
Oct 30 06:24:14.565774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:14.565695    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"]
Oct 30 06:24:14.574091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:14.574067    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:14.579909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:14.579885    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-128436-openshift-default-fd9c97c99-ttzks"]
Oct 30 06:24:14.737273 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:14.737222    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:16.063166 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:16.063127    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:16.401769 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:16.401690    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" path="/var/lib/kubelet/pods/d92910fe-e3b0-41f6-92f3-aee750b56067/volumes"
Oct 30 06:24:18.052457 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:18.052392    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:19.778373 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:19.778333    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:22.240571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:22.240537    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:24.030939 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:24.030912    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:25.829454 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:25.829422    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:27.539879 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:27.539848    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:27.727536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:27.727490    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:27.897260 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:27.897180    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:29.201532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:29.201498    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:29.704810 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:29.704776    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:29.867646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:29.867615    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:30.503716 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:30.503683    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:32.016910 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:32.016879    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:32.552256 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:24:32.552225    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:24:33.692901 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:33.692857    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:33.858900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:33.858839    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:34.786308 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:34.786280    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:35.454721 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:35.454691    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:37.063220 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.063186    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:37.222145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.222112    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:37.441532 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.441498    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:37.883582 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.883468    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:37.897384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.897354    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"]
Oct 30 06:24:37.897959 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.897930    5066 cpu_manager.go:414] "containerMap: removing container" podUID="eda6dbe1-e406-4d55-a0e2-1c532edda522" containerName="istio-proxy"
Oct 30 06:24:37.897959 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.897951    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eda6dbe1-e406-4d55-a0e2-1c532edda522" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.897978    5066 cpu_manager.go:414] "containerMap: removing container" podUID="ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.897985    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.897995    5066 cpu_manager.go:414] "containerMap: removing container" podUID="c7295343-bb58-43a6-a460-9b26fb26edc0" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898003    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7295343-bb58-43a6-a460-9b26fb26edc0" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898015    5066 cpu_manager.go:414] "containerMap: removing container" podUID="3289c892-df28-4817-aef4-2291dc0d3a4b" containerName="maas-api"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898022    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3289c892-df28-4817-aef4-2291dc0d3a4b" containerName="maas-api"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898031    5066 cpu_manager.go:414] "containerMap: removing container" podUID="f66efc60-7d24-4463-8668-6af9c0951b77" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898037    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f66efc60-7d24-4463-8668-6af9c0951b77" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898053    5066 cpu_manager.go:414] "containerMap: removing container" podUID="b1193647-74ea-437e-b1ae-9040d7440d48" containerName="maas-api"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898060    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1193647-74ea-437e-b1ae-9040d7440d48" containerName="maas-api"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898070    5066 cpu_manager.go:414] "containerMap: removing container" podUID="2a6d1b0d-6486-479c-a878-93cb39839013" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898079    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a6d1b0d-6486-479c-a878-93cb39839013" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898086    5066 cpu_manager.go:414] "containerMap: removing container" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898093    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898104    5066 cpu_manager.go:414] "containerMap: removing container" podUID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898111    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898121    5066 cpu_manager.go:414] "containerMap: removing container" podUID="4cf69466-534b-4646-99ba-f1bbb28fb92b" containerName="maas-api"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898128    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cf69466-534b-4646-99ba-f1bbb28fb92b" containerName="maas-api"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898137    5066 cpu_manager.go:414] "containerMap: removing container" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898144    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerName="istio-proxy"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898157    5066 cpu_manager.go:414] "containerMap: removing container" podUID="b64460f3-6d61-455e-923c-aa6e46ffca11" containerName="maas-api"
Oct 30 06:24:37.898157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898164    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b64460f3-6d61-455e-923c-aa6e46ffca11" containerName="maas-api"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898175    5066 cpu_manager.go:414] "containerMap: removing container" podUID="ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9" containerName="maas-api"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898181    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9" containerName="maas-api"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898206    5066 cpu_manager.go:414] "containerMap: removing container" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898213    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898221    5066 cpu_manager.go:414] "containerMap: removing container" podUID="dd898281-6242-49c8-a18b-f8d01d36863f" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898228    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd898281-6242-49c8-a18b-f8d01d36863f" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898242    5066 cpu_manager.go:414] "containerMap: removing container" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898249    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898272    5066 cpu_manager.go:414] "containerMap: removing container" podUID="e977c71a-0612-4f50-876e-720196d43303" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898280    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e977c71a-0612-4f50-876e-720196d43303" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898292    5066 cpu_manager.go:414] "containerMap: removing container" podUID="8a1d2da0-194d-44b7-94ad-c49bd3cb2451" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898298    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a1d2da0-194d-44b7-94ad-c49bd3cb2451" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898313    5066 cpu_manager.go:414] "containerMap: removing container" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898320    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898335    5066 cpu_manager.go:414] "containerMap: removing container" podUID="9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898341    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898358    5066 cpu_manager.go:414] "containerMap: removing container" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898365    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898377    5066 cpu_manager.go:414] "containerMap: removing container" podUID="f575bda0-bead-41dc-a5e5-d633cad8e59d" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898384    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f575bda0-bead-41dc-a5e5-d633cad8e59d" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898400    5066 cpu_manager.go:414] "containerMap: removing container" podUID="43685c17-86ec-4839-bb32-7251aba4e869" containerName="payload-processing"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898407    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43685c17-86ec-4839-bb32-7251aba4e869" containerName="payload-processing"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898419    5066 cpu_manager.go:414] "containerMap: removing container" podUID="c98c4321-bd50-417b-ae05-8def39068949" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898426    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c98c4321-bd50-417b-ae05-8def39068949" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898439    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898445    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898459    5066 cpu_manager.go:414] "containerMap: removing container" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898466    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898477    5066 cpu_manager.go:414] "containerMap: removing container" podUID="aaf0009d-4301-46f7-9427-47e17c7a47d6" containerName="maas-api"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898484    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aaf0009d-4301-46f7-9427-47e17c7a47d6" containerName="maas-api"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898624    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="5edbb8c6-de2c-4167-8e45-ba6c78204165" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898641    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd898281-6242-49c8-a18b-f8d01d36863f" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898650    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7ae4c02-ca60-4ac0-b467-7be612e7faa1" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898661    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="e977c71a-0612-4f50-876e-720196d43303" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898671    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a1d2da0-194d-44b7-94ad-c49bd3cb2451" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898681    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="71a86815-9e7e-4c32-a865-e51a1c51c7c9" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898691    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fd717f7-573f-4bc8-99b6-31ccdaa6d45d" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898700    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="40749c08-d2c2-4a79-acb6-45b23d69f7db" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898709    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="f575bda0-bead-41dc-a5e5-d633cad8e59d" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898718    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="43685c17-86ec-4839-bb32-7251aba4e869" containerName="payload-processing"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898727    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="c98c4321-bd50-417b-ae05-8def39068949" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898736    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5d6f936-0eb8-4937-b596-497662f44fbc" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898747    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="e74e2be3-9336-4f70-81bf-e7caba8d1a8d" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898755    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="aaf0009d-4301-46f7-9427-47e17c7a47d6" containerName="maas-api"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898776    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="eda6dbe1-e406-4d55-a0e2-1c532edda522" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898784    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec1fd3a3-dabc-4251-8ea3-c30bf2302c6d" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898792    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7295343-bb58-43a6-a460-9b26fb26edc0" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898801    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="3289c892-df28-4817-aef4-2291dc0d3a4b" containerName="maas-api"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898810    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="f66efc60-7d24-4463-8668-6af9c0951b77" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898820    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1193647-74ea-437e-b1ae-9040d7440d48" containerName="maas-api"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898829    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a6d1b0d-6486-479c-a878-93cb39839013" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898837    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="61de80c4-337b-4223-9a80-92781d29ff3b" containerName="main"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898845    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="53b1f7d3-1fde-456e-92f1-8f45c89036ae" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898855    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cf69466-534b-4646-99ba-f1bbb28fb92b" containerName="maas-api"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898865    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="d92910fe-e3b0-41f6-92f3-aee750b56067" containerName="istio-proxy"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898875    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="b64460f3-6d61-455e-923c-aa6e46ffca11" containerName="maas-api"
Oct 30 06:24:37.899368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.898884    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed3cb7a5-cf23-4257-89b4-5b693d2b2ef9" containerName="maas-api"
Oct 30 06:24:37.902468 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.902449    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:37.920634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:37.920573    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"]
Oct 30 06:24:38.002390 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.002359    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 06:24:38.002564 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.002472    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:24:38.006142 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.006116    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5\" (UID: \"32a02a07-cb03-4262-b912-27a8b26b7c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:38.006293 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.006271    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m9h8\" (UniqueName: \"kubernetes.io/projected/32a02a07-cb03-4262-b912-27a8b26b7c50-kube-api-access-7m9h8\") pod \"maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5\" (UID: \"32a02a07-cb03-4262-b912-27a8b26b7c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:38.006365 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.006350    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5\" (UID: \"32a02a07-cb03-4262-b912-27a8b26b7c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:38.108348 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.108322    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m9h8\" (UniqueName: \"kubernetes.io/projected/32a02a07-cb03-4262-b912-27a8b26b7c50-kube-api-access-7m9h8\") pod \"maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5\" (UID: \"32a02a07-cb03-4262-b912-27a8b26b7c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:38.108745 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.108370    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5\" (UID: \"32a02a07-cb03-4262-b912-27a8b26b7c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:38.108745 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.108474    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5\" (UID: \"32a02a07-cb03-4262-b912-27a8b26b7c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:38.109177 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.109085    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5\" (UID: \"32a02a07-cb03-4262-b912-27a8b26b7c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:38.111138 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.111116    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5\" (UID: \"32a02a07-cb03-4262-b912-27a8b26b7c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:38.133564 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.133534    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m9h8\" (UniqueName: \"kubernetes.io/projected/32a02a07-cb03-4262-b912-27a8b26b7c50-kube-api-access-7m9h8\") pod \"maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5\" (UID: \"32a02a07-cb03-4262-b912-27a8b26b7c50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:38.215767 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.215727    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:38.522933 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.522902    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"]
Oct 30 06:24:38.524503 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:24:38.524472    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a02a07_cb03_4262_b912_27a8b26b7c50.slice/crio-05e5f3fee806b959c26ffa49fcb40f52073dc6f8e7cabdde39c157f8eca7875c WatchSource:0}: Error finding container 05e5f3fee806b959c26ffa49fcb40f52073dc6f8e7cabdde39c157f8eca7875c: Status 404 returned error can't find the container with id 05e5f3fee806b959c26ffa49fcb40f52073dc6f8e7cabdde39c157f8eca7875c
Oct 30 06:24:38.649107 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:38.649068    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5" event={"ID":"32a02a07-cb03-4262-b912-27a8b26b7c50","Type":"ContainerStarted","Data":"05e5f3fee806b959c26ffa49fcb40f52073dc6f8e7cabdde39c157f8eca7875c"}
Oct 30 06:24:39.236446 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:39.236408    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:39.412029 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:39.411997    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:39.658831 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:39.658733    5066 generic.go:356] "Generic (PLEG): container finished" podID="32a02a07-cb03-4262-b912-27a8b26b7c50" containerID="72672ead92272af0557d83dab390d5f95ef743c453bde70b768dbe21a676a5e3" exitCode=0
Oct 30 06:24:39.658831 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:39.658777    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5" event={"ID":"32a02a07-cb03-4262-b912-27a8b26b7c50","Type":"ContainerDied","Data":"72672ead92272af0557d83dab390d5f95ef743c453bde70b768dbe21a676a5e3"}
Oct 30 06:24:40.333489 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:40.333451    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:40.818373 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:40.818349    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:40.835983 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:40.835953    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32a02a07-cb03-4262-b912-27a8b26b7c50-kube-api-access-7m9h8\" (UniqueName: \"kubernetes.io/projected/32a02a07-cb03-4262-b912-27a8b26b7c50-kube-api-access-7m9h8\") pod \"32a02a07-cb03-4262-b912-27a8b26b7c50\" (UID: \"32a02a07-cb03-4262-b912-27a8b26b7c50\") "
Oct 30 06:24:40.836145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:40.836006    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-openshift-ca\") pod \"32a02a07-cb03-4262-b912-27a8b26b7c50\" (UID: \"32a02a07-cb03-4262-b912-27a8b26b7c50\") "
Oct 30 06:24:40.836224 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:40.836153    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-xks-ca\") pod \"32a02a07-cb03-4262-b912-27a8b26b7c50\" (UID: \"32a02a07-cb03-4262-b912-27a8b26b7c50\") "
Oct 30 06:24:40.836416 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:40.836391    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-openshift-ca" pod "32a02a07-cb03-4262-b912-27a8b26b7c50" (UID: "32a02a07-cb03-4262-b912-27a8b26b7c50"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:24:40.838277 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:40.838253    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-xks-ca" pod "32a02a07-cb03-4262-b912-27a8b26b7c50" (UID: "32a02a07-cb03-4262-b912-27a8b26b7c50"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:24:40.838376 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:40.838294    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32a02a07-cb03-4262-b912-27a8b26b7c50-kube-api-access-7m9h8" pod "32a02a07-cb03-4262-b912-27a8b26b7c50" (UID: "32a02a07-cb03-4262-b912-27a8b26b7c50"). InnerVolumeSpecName "kube-api-access-7m9h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:24:40.937076 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:40.936994    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:40.937076 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:40.937024    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7m9h8\" (UniqueName: \"kubernetes.io/projected/32a02a07-cb03-4262-b912-27a8b26b7c50-kube-api-access-7m9h8\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:40.937076 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:40.937033    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32a02a07-cb03-4262-b912-27a8b26b7c50-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:24:41.670798 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:41.670768    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"
Oct 30 06:24:41.670798 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:41.670775    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5" event={"ID":"32a02a07-cb03-4262-b912-27a8b26b7c50","Type":"ContainerDied","Data":"05e5f3fee806b959c26ffa49fcb40f52073dc6f8e7cabdde39c157f8eca7875c"}
Oct 30 06:24:41.671295 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:41.670818    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05e5f3fee806b959c26ffa49fcb40f52073dc6f8e7cabdde39c157f8eca7875c"
Oct 30 06:24:42.153671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:42.153650    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:43.924361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:43.924324    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:45.766952 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:45.766920    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:47.498528 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:47.498449    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:48.484378 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:48.484341    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:48.649320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:48.649283    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:49.053048 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:49.053020    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:49.763201 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:49.763166    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:52.157218 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:52.157184    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:56.496862 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:56.496825    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:56.668106 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:56.668076    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:58.071546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:58.071510    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:24:59.847827 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:24:59.847788    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:01.961646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:01.961610    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:02.553704 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:25:02.553672    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:25:03.975260 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:03.975229    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:04.064257 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:04.064225    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"]
Oct 30 06:25:04.078794 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:04.078768    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w4-cc33d3-xfkb5"]
Oct 30 06:25:04.142421 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:04.142389    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:04.304183 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:04.304100    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:04.402203 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:04.402163    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32a02a07-cb03-4262-b912-27a8b26b7c50" path="/var/lib/kubelet/pods/32a02a07-cb03-4262-b912-27a8b26b7c50/volumes"
Oct 30 06:25:04.656357 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:04.656329    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:06.803544 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:06.803513    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:09.176823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:09.176793    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:11.073181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:11.073145    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:11.534711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:11.534676    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:11.836101 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:11.836002    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:11.908369 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:11.908337    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"]
Oct 30 06:25:11.908621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:11.908555    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" podUID="05d2bec0-3663-4c53-8b93-b0bb50191185" containerName="istio-proxy" containerID="cri-o://33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74" gracePeriod=30
Oct 30 06:25:12.044007 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:12.043971    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:12.488362 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:12.488331    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:13.404858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:13.404812    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:14.095776 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:14.095743    5066 patch_prober.go:29] interesting pod/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:25:14.096164 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:14.095799    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" podUID="05d2bec0-3663-4c53-8b93-b0bb50191185" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:25:14.919434 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:14.919404    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:15.096745 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:15.096706    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:15.375016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:15.374930    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:17.198469 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.198441    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:25:17.246568 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.246540    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-socket\") pod \"05d2bec0-3663-4c53-8b93-b0bb50191185\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") "
Oct 30 06:25:17.246787 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.246643    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-kube-api-access-wffzd\" (UniqueName: \"kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-kube-api-access-wffzd\") pod \"05d2bec0-3663-4c53-8b93-b0bb50191185\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") "
Oct 30 06:25:17.246787 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.246676    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-podinfo\") pod \"05d2bec0-3663-4c53-8b93-b0bb50191185\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") "
Oct 30 06:25:17.246787 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.246718    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-credential-socket\") pod \"05d2bec0-3663-4c53-8b93-b0bb50191185\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") "
Oct 30 06:25:17.246787 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.246772    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05d2bec0-3663-4c53-8b93-b0bb50191185-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05d2bec0-3663-4c53-8b93-b0bb50191185-istiod-ca-cert\") pod \"05d2bec0-3663-4c53-8b93-b0bb50191185\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") "
Oct 30 06:25:17.247036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.246806    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-token\" (UniqueName: \"kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-token\") pod \"05d2bec0-3663-4c53-8b93-b0bb50191185\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") "
Oct 30 06:25:17.247036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.246880    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-certs\") pod \"05d2bec0-3663-4c53-8b93-b0bb50191185\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") "
Oct 30 06:25:17.247036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.246911    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-envoy\") pod \"05d2bec0-3663-4c53-8b93-b0bb50191185\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") "
Oct 30 06:25:17.247036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.246965    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-socket" pod "05d2bec0-3663-4c53-8b93-b0bb50191185" (UID: "05d2bec0-3663-4c53-8b93-b0bb50191185"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:17.247036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.246994    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-data\") pod \"05d2bec0-3663-4c53-8b93-b0bb50191185\" (UID: \"05d2bec0-3663-4c53-8b93-b0bb50191185\") "
Oct 30 06:25:17.247309 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.247214    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-certs" pod "05d2bec0-3663-4c53-8b93-b0bb50191185" (UID: "05d2bec0-3663-4c53-8b93-b0bb50191185"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:17.247457 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.247426    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-data" pod "05d2bec0-3663-4c53-8b93-b0bb50191185" (UID: "05d2bec0-3663-4c53-8b93-b0bb50191185"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:17.247630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.247543    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:17.247630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.247564    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:17.247956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.247930    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05d2bec0-3663-4c53-8b93-b0bb50191185-istiod-ca-cert" pod "05d2bec0-3663-4c53-8b93-b0bb50191185" (UID: "05d2bec0-3663-4c53-8b93-b0bb50191185"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:25:17.248086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.248011    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-credential-socket" pod "05d2bec0-3663-4c53-8b93-b0bb50191185" (UID: "05d2bec0-3663-4c53-8b93-b0bb50191185"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:17.250221 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.250197    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-podinfo" pod "05d2bec0-3663-4c53-8b93-b0bb50191185" (UID: "05d2bec0-3663-4c53-8b93-b0bb50191185"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:25:17.250221 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.250214    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-kube-api-access-wffzd" pod "05d2bec0-3663-4c53-8b93-b0bb50191185" (UID: "05d2bec0-3663-4c53-8b93-b0bb50191185"). InnerVolumeSpecName "kube-api-access-wffzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:25:17.250483 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.250463    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-envoy" pod "05d2bec0-3663-4c53-8b93-b0bb50191185" (UID: "05d2bec0-3663-4c53-8b93-b0bb50191185"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:17.251379 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.251341    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-token" pod "05d2bec0-3663-4c53-8b93-b0bb50191185" (UID: "05d2bec0-3663-4c53-8b93-b0bb50191185"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:25:17.267043 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.267015    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:17.348571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.348535    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wffzd\" (UniqueName: \"kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-kube-api-access-wffzd\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:17.348571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.348568    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:17.348571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.348577    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:17.348571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.348598    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05d2bec0-3663-4c53-8b93-b0bb50191185-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:17.348571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.348607    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:17.348903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.348615    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:17.348903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.348623    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05d2bec0-3663-4c53-8b93-b0bb50191185-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:17.395505 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.390302    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"]
Oct 30 06:25:17.424654 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.424526    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"]
Oct 30 06:25:17.613234 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.613197    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:17.873188 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.873088    5066 generic.go:356] "Generic (PLEG): container finished" podID="05d2bec0-3663-4c53-8b93-b0bb50191185" containerID="33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74" exitCode=0
Oct 30 06:25:17.873188 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.873185    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"
Oct 30 06:25:17.873423 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.873187    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" event={"ID":"05d2bec0-3663-4c53-8b93-b0bb50191185","Type":"ContainerDied","Data":"33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74"}
Oct 30 06:25:17.873423 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.873258    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx" event={"ID":"05d2bec0-3663-4c53-8b93-b0bb50191185","Type":"ContainerDied","Data":"df08d2fc83a516dc454044f8a8e95f19ec33f155533f958a581c9361e4585fa1"}
Oct 30 06:25:17.873423 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.873287    5066 scope.go:122] "RemoveContainer" containerID="33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74"
Oct 30 06:25:17.883898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.883883    5066 scope.go:122] "RemoveContainer" containerID="33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74"
Oct 30 06:25:17.884207 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:25:17.884185    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74\": container with ID starting with 33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74 not found: ID does not exist" containerID="33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74"
Oct 30 06:25:17.884289 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.884216    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74"} err="failed to get container status \"33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74\": rpc error: code = NotFound desc = could not find container \"33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74\": container with ID starting with 33c40cb61e0802cf737302c2899309f0c41ba12e3b9b498cd91f4ade5e9cba74 not found: ID does not exist"
Oct 30 06:25:17.938724 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.938690    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"]
Oct 30 06:25:17.943354 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:17.943330    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-cc33d3-openshift-default-6d67448675-kk6sx"]
Oct 30 06:25:18.401541 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:18.401511    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05d2bec0-3663-4c53-8b93-b0bb50191185" path="/var/lib/kubelet/pods/05d2bec0-3663-4c53-8b93-b0bb50191185/volumes"
Oct 30 06:25:18.875062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:18.875024    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:19.102390 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:19.102346    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:19.294390 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:19.294345    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:19.549790 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:19.549711    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:19.720664 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:19.720635    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:20.331814 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:20.331777    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:22.581977 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:22.581947    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:23.199411 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:23.199357    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:23.379967 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:23.379932    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:24.336367 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:24.336337    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:26.212766 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:26.212700    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:26.733320 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:26.733279    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:26.911565 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:26.911527    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:27.088192 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:27.088107    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:28.142851 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:28.142816    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:29.975111 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:29.975082    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:31.992109 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:31.992069    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:32.419518 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.419482    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main" containerID="cri-o://76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8" gracePeriod=45
Oct 30 06:25:32.445565 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.445518    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main" containerID="cri-o://cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f" gracePeriod=45
Oct 30 06:25:32.552187 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:25:32.552124    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:25:32.713164 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.713137    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 06:25:32.744777 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.744734    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 06:25:32.820323 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820290    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-dshm\") pod \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") "
Oct 30 06:25:32.820323 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820325    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-tmp-dir\") pod \"ef3506e3-2464-46ed-8808-3c5d681330ad\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") "
Oct 30 06:25:32.820635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820381    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tmp-dir\") pod \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") "
Oct 30 06:25:32.820635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820401    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8bdc1785-b6ce-4de9-bda2-6127f1964b13-kube-api-access-j5xkg\" (UniqueName: \"kubernetes.io/projected/8bdc1785-b6ce-4de9-bda2-6127f1964b13-kube-api-access-j5xkg\") pod \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") "
Oct 30 06:25:32.820635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820422    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-dshm\") pod \"ef3506e3-2464-46ed-8808-3c5d681330ad\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") "
Oct 30 06:25:32.820635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820477    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-home\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-home\") pod \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") "
Oct 30 06:25:32.820635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820509    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tls-certs\") pod \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") "
Oct 30 06:25:32.820635 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820565    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-model-cache\") pod \"ef3506e3-2464-46ed-8808-3c5d681330ad\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") "
Oct 30 06:25:32.821054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820639    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef3506e3-2464-46ed-8808-3c5d681330ad-kube-api-access-lb2vh\" (UniqueName: \"kubernetes.io/projected/ef3506e3-2464-46ed-8808-3c5d681330ad-kube-api-access-lb2vh\") pod \"ef3506e3-2464-46ed-8808-3c5d681330ad\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") "
Oct 30 06:25:32.821054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820709    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef3506e3-2464-46ed-8808-3c5d681330ad-tls-certs\" (UniqueName: \"kubernetes.io/secret/ef3506e3-2464-46ed-8808-3c5d681330ad-tls-certs\") pod \"ef3506e3-2464-46ed-8808-3c5d681330ad\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") "
Oct 30 06:25:32.821054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820711    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tmp-dir" pod "8bdc1785-b6ce-4de9-bda2-6127f1964b13" (UID: "8bdc1785-b6ce-4de9-bda2-6127f1964b13"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:32.821054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820720    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-tmp-dir" pod "ef3506e3-2464-46ed-8808-3c5d681330ad" (UID: "ef3506e3-2464-46ed-8808-3c5d681330ad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:32.821054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820753    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-model-cache\") pod \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\" (UID: \"8bdc1785-b6ce-4de9-bda2-6127f1964b13\") "
Oct 30 06:25:32.821054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820818    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-home\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-home\") pod \"ef3506e3-2464-46ed-8808-3c5d681330ad\" (UID: \"ef3506e3-2464-46ed-8808-3c5d681330ad\") "
Oct 30 06:25:32.821054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820926    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-model-cache" pod "ef3506e3-2464-46ed-8808-3c5d681330ad" (UID: "ef3506e3-2464-46ed-8808-3c5d681330ad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:32.821054 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.820930    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-home" pod "8bdc1785-b6ce-4de9-bda2-6127f1964b13" (UID: "8bdc1785-b6ce-4de9-bda2-6127f1964b13"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:32.821533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.821136    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-model-cache" pod "8bdc1785-b6ce-4de9-bda2-6127f1964b13" (UID: "8bdc1785-b6ce-4de9-bda2-6127f1964b13"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:32.821533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.821263    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-home" pod "ef3506e3-2464-46ed-8808-3c5d681330ad" (UID: "ef3506e3-2464-46ed-8808-3c5d681330ad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:32.821533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.821515    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:32.821889 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.821535    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:32.821889 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.821551    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:32.821889 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.821565    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:32.821889 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.821577    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:32.821889 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.821605    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:32.823675 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.823652    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bdc1785-b6ce-4de9-bda2-6127f1964b13-kube-api-access-j5xkg" pod "8bdc1785-b6ce-4de9-bda2-6127f1964b13" (UID: "8bdc1785-b6ce-4de9-bda2-6127f1964b13"). InnerVolumeSpecName "kube-api-access-j5xkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:25:32.823813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.823687    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef3506e3-2464-46ed-8808-3c5d681330ad-kube-api-access-lb2vh" pod "ef3506e3-2464-46ed-8808-3c5d681330ad" (UID: "ef3506e3-2464-46ed-8808-3c5d681330ad"). InnerVolumeSpecName "kube-api-access-lb2vh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:25:32.823813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.823786    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tls-certs" pod "8bdc1785-b6ce-4de9-bda2-6127f1964b13" (UID: "8bdc1785-b6ce-4de9-bda2-6127f1964b13"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:25:32.823813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.823790    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef3506e3-2464-46ed-8808-3c5d681330ad-tls-certs" pod "ef3506e3-2464-46ed-8808-3c5d681330ad" (UID: "ef3506e3-2464-46ed-8808-3c5d681330ad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:25:32.823955 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.823938    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-dshm" pod "ef3506e3-2464-46ed-8808-3c5d681330ad" (UID: "ef3506e3-2464-46ed-8808-3c5d681330ad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:32.825831 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.825812    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-dshm" pod "8bdc1785-b6ce-4de9-bda2-6127f1964b13" (UID: "8bdc1785-b6ce-4de9-bda2-6127f1964b13"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:25:32.922844 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.922806    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5xkg\" (UniqueName: \"kubernetes.io/projected/8bdc1785-b6ce-4de9-bda2-6127f1964b13-kube-api-access-j5xkg\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:32.922844 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.922838    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef3506e3-2464-46ed-8808-3c5d681330ad-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:32.922844 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.922851    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bdc1785-b6ce-4de9-bda2-6127f1964b13-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:32.923111 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.922865    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb2vh\" (UniqueName: \"kubernetes.io/projected/ef3506e3-2464-46ed-8808-3c5d681330ad-kube-api-access-lb2vh\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:32.923111 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.922875    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef3506e3-2464-46ed-8808-3c5d681330ad-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:32.923111 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.922886    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bdc1785-b6ce-4de9-bda2-6127f1964b13-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:25:32.957877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.957790    5066 generic.go:356] "Generic (PLEG): container finished" podID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerID="76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8" exitCode=0
Oct 30 06:25:32.957877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.957831    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"
Oct 30 06:25:32.957877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.957865    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" event={"ID":"8bdc1785-b6ce-4de9-bda2-6127f1964b13","Type":"ContainerDied","Data":"76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8"}
Oct 30 06:25:32.958146 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.957901    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr" event={"ID":"8bdc1785-b6ce-4de9-bda2-6127f1964b13","Type":"ContainerDied","Data":"d1d3a1adc335f3c4bb84cf3bb2156a3c06e6327c8088604af1ea589f100c7ec3"}
Oct 30 06:25:32.958146 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.957918    5066 scope.go:122] "RemoveContainer" containerID="76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8"
Oct 30 06:25:32.959397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.959375    5066 generic.go:356] "Generic (PLEG): container finished" podID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerID="cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f" exitCode=0
Oct 30 06:25:32.959494 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.959405    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"
Oct 30 06:25:32.959494 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.959429    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" event={"ID":"ef3506e3-2464-46ed-8808-3c5d681330ad","Type":"ContainerDied","Data":"cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f"}
Oct 30 06:25:32.959494 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.959458    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8" event={"ID":"ef3506e3-2464-46ed-8808-3c5d681330ad","Type":"ContainerDied","Data":"9e0930cfda20c51b713fb55cfcff01327089f89f084525f9e6d156170104cb1c"}
Oct 30 06:25:32.969092 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.969069    5066 scope.go:122] "RemoveContainer" containerID="76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8"
Oct 30 06:25:32.969657 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:25:32.969579    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8\": container with ID starting with 76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8 not found: ID does not exist" containerID="76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8"
Oct 30 06:25:32.969766 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.969667    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8"} err="failed to get container status \"76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8\": rpc error: code = NotFound desc = could not find container \"76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8\": container with ID starting with 76460fa2e4f9920a06d6c0c9089fb0b8a1966989bb71a8a8f6f114ce321c12e8 not found: ID does not exist"
Oct 30 06:25:32.969766 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.969701    5066 scope.go:122] "RemoveContainer" containerID="cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f"
Oct 30 06:25:32.983097 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.983072    5066 scope.go:122] "RemoveContainer" containerID="cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f"
Oct 30 06:25:32.983449 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:25:32.983424    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f\": container with ID starting with cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f not found: ID does not exist" containerID="cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f"
Oct 30 06:25:32.983521 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:32.983459    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f"} err="failed to get container status \"cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f\": rpc error: code = NotFound desc = could not find container \"cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f\": container with ID starting with cb7b431e3016b103127d38af014f09ec5f36242efc0006c4e59e782276c48d9f not found: ID does not exist"
Oct 30 06:25:33.035077 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:33.035040    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"]
Oct 30 06:25:33.048546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:33.048513    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e2e3d0/simulator-w4-e2e3d0-kserve-75b67f5b8b-gqfn8"]
Oct 30 06:25:33.087383 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:33.087352    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"]
Oct 30 06:25:33.094863 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:33.094834    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e2e3d0/distinct-2-w4-e2e3d0-kserve-5b6b84ccdf-s45gr"]
Oct 30 06:25:33.991217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:33.991179    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:34.401579 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:34.401495    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" path="/var/lib/kubelet/pods/8bdc1785-b6ce-4de9-bda2-6127f1964b13/volumes"
Oct 30 06:25:34.402124 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:34.402104    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" path="/var/lib/kubelet/pods/ef3506e3-2464-46ed-8808-3c5d681330ad/volumes"
Oct 30 06:25:34.446683 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:34.446647    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:34.915011 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:34.914979    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:36.367231 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:36.367196    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:38.083636 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:38.083575    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:39.788506 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:39.788474    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:41.288134 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:41.288100    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:41.485346 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:41.485317    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:41.655447 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:41.655414    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:42.026006 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:42.025921    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:42.194813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:42.194777    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:42.754637 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:42.754606    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:44.233911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:44.233869    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:46.154225 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:46.154193    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:47.778850 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:47.778805    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:48.336937 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:48.336909    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:48.536411 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:48.536379    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:48.959759 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:48.959728    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:49.838629 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:49.838568    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:50.082157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:50.082105    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:50.351802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:50.351771    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:50.624643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:50.624519    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:52.437831 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:52.437799    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:54.362279 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:54.362249    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:56.087960 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:56.087911    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:56.407004 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:56.406968    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:56.654974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:56.654933    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:56.844025 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:56.843946    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:57.109641 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:57.109542    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:57.918641 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:57.918565    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:58.100905 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:58.100872    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:58.218089 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:58.218007    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"]
Oct 30 06:25:58.218502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:58.218268    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" podUID="dafc9b85-538c-4f66-8578-012bad182b0a" containerName="istio-proxy" containerID="cri-o://d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849" gracePeriod=30
Oct 30 06:25:58.258801 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:58.258763    5066 patch_prober.go:29] interesting pod/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 06:25:58.258972 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:58.258835    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" podUID="dafc9b85-538c-4f66-8578-012bad182b0a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 06:25:58.320960 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:58.320927    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:25:59.910071 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:25:59.910034    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:01.357816 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:01.357778    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:01.533128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:01.533091    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:01.871843 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:01.871807    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:02.552260 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:26:02.552218    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:26:03.364663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.364629    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:03.490634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.490536    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 06:26:03.584164 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.584134    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:03.644661 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.644634    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dafc9b85-538c-4f66-8578-012bad182b0a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dafc9b85-538c-4f66-8578-012bad182b0a-istio-podinfo\") pod \"dafc9b85-538c-4f66-8578-012bad182b0a\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") "
Oct 30 06:26:03.644850 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.644685    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-socket\") pod \"dafc9b85-538c-4f66-8578-012bad182b0a\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") "
Oct 30 06:26:03.644903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.644858    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-certs\") pod \"dafc9b85-538c-4f66-8578-012bad182b0a\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") "
Oct 30 06:26:03.644903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.644898    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-kube-api-access-dz48z\" (UniqueName: \"kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-kube-api-access-dz48z\") pod \"dafc9b85-538c-4f66-8578-012bad182b0a\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") "
Oct 30 06:26:03.645020 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.644938    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-istio-token\" (UniqueName: \"kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-istio-token\") pod \"dafc9b85-538c-4f66-8578-012bad182b0a\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") "
Oct 30 06:26:03.645020 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.644962    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-envoy\") pod \"dafc9b85-538c-4f66-8578-012bad182b0a\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") "
Oct 30 06:26:03.645020 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.644938    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-socket" pod "dafc9b85-538c-4f66-8578-012bad182b0a" (UID: "dafc9b85-538c-4f66-8578-012bad182b0a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:03.645204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.645044    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dafc9b85-538c-4f66-8578-012bad182b0a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dafc9b85-538c-4f66-8578-012bad182b0a-istiod-ca-cert\") pod \"dafc9b85-538c-4f66-8578-012bad182b0a\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") "
Oct 30 06:26:03.645204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.645128    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-certs" pod "dafc9b85-538c-4f66-8578-012bad182b0a" (UID: "dafc9b85-538c-4f66-8578-012bad182b0a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:03.645204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.645186    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-data\") pod \"dafc9b85-538c-4f66-8578-012bad182b0a\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") "
Oct 30 06:26:03.645349 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.645225    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-credential-socket\") pod \"dafc9b85-538c-4f66-8578-012bad182b0a\" (UID: \"dafc9b85-538c-4f66-8578-012bad182b0a\") "
Oct 30 06:26:03.645438 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.645409    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-data" pod "dafc9b85-538c-4f66-8578-012bad182b0a" (UID: "dafc9b85-538c-4f66-8578-012bad182b0a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:03.645687 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.645659    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dafc9b85-538c-4f66-8578-012bad182b0a-istiod-ca-cert" pod "dafc9b85-538c-4f66-8578-012bad182b0a" (UID: "dafc9b85-538c-4f66-8578-012bad182b0a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:26:03.645687 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.645668    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-credential-socket" pod "dafc9b85-538c-4f66-8578-012bad182b0a" (UID: "dafc9b85-538c-4f66-8578-012bad182b0a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:03.645824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.645684    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:03.645824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.645704    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:03.645824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.645716    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:03.647179 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.647159    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dafc9b85-538c-4f66-8578-012bad182b0a-istio-podinfo" pod "dafc9b85-538c-4f66-8578-012bad182b0a" (UID: "dafc9b85-538c-4f66-8578-012bad182b0a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:26:03.647313 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.647294    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-envoy" pod "dafc9b85-538c-4f66-8578-012bad182b0a" (UID: "dafc9b85-538c-4f66-8578-012bad182b0a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:03.647745 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.647721    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-istio-token" pod "dafc9b85-538c-4f66-8578-012bad182b0a" (UID: "dafc9b85-538c-4f66-8578-012bad182b0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:26:03.647820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.647727    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-kube-api-access-dz48z" pod "dafc9b85-538c-4f66-8578-012bad182b0a" (UID: "dafc9b85-538c-4f66-8578-012bad182b0a"). InnerVolumeSpecName "kube-api-access-dz48z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:26:03.746858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.746775    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:03.746858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.746802    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dafc9b85-538c-4f66-8578-012bad182b0a-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:03.746858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.746812    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dz48z\" (UniqueName: \"kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-kube-api-access-dz48z\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:03.746858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.746821    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dafc9b85-538c-4f66-8578-012bad182b0a-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:03.746858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.746830    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dafc9b85-538c-4f66-8578-012bad182b0a-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:03.746858 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:03.746837    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dafc9b85-538c-4f66-8578-012bad182b0a-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:04.128642 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:04.128546    5066 generic.go:356] "Generic (PLEG): container finished" podID="dafc9b85-538c-4f66-8578-012bad182b0a" containerID="d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849" exitCode=0
Oct 30 06:26:04.128642 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:04.128615    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"
Oct 30 06:26:04.128642 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:04.128629    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" event={"ID":"dafc9b85-538c-4f66-8578-012bad182b0a","Type":"ContainerDied","Data":"d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849"}
Oct 30 06:26:04.128884 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:04.128692    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw" event={"ID":"dafc9b85-538c-4f66-8578-012bad182b0a","Type":"ContainerDied","Data":"537c7a025a41e0b6668127eac9d2c095f5a0e6c916c6443578b318ecfccc9508"}
Oct 30 06:26:04.128884 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:04.128722    5066 scope.go:122] "RemoveContainer" containerID="d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849"
Oct 30 06:26:04.139024 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:04.139006    5066 scope.go:122] "RemoveContainer" containerID="d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849"
Oct 30 06:26:04.139320 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:26:04.139299    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849\": container with ID starting with d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849 not found: ID does not exist" containerID="d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849"
Oct 30 06:26:04.139401 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:04.139329    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849"} err="failed to get container status \"d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849\": rpc error: code = NotFound desc = could not find container \"d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849\": container with ID starting with d1399ae6fd28c1a24e37794fce6a535786310ab7ca44e42d1f1f548496815849 not found: ID does not exist"
Oct 30 06:26:04.205450 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:04.205391    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"]
Oct 30 06:26:04.219774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:04.219741    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e2e3d0-openshift-default-57898b578c-hnpsw"]
Oct 30 06:26:04.402263 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:04.402188    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dafc9b85-538c-4f66-8578-012bad182b0a" path="/var/lib/kubelet/pods/dafc9b85-538c-4f66-8578-012bad182b0a/volumes"
Oct 30 06:26:05.056700 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.056666    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"]
Oct 30 06:26:05.057225 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.056924    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m" podUID="552a0155-ae0c-4811-b0ab-d11480e9cbd4" containerName="maas-api" containerID="cri-o://04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852" gracePeriod=30
Oct 30 06:26:05.328016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.327990    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:26:05.344218 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.344188    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:05.470279 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.470245    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-maas-api-tls\") pod \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") "
Oct 30 06:26:05.470482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.470313    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/552a0155-ae0c-4811-b0ab-d11480e9cbd4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/552a0155-ae0c-4811-b0ab-d11480e9cbd4-openshift-service-ca\") pod \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") "
Oct 30 06:26:05.470482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.470421    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-metrics-certs\") pod \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") "
Oct 30 06:26:05.470482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.470459    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/552a0155-ae0c-4811-b0ab-d11480e9cbd4-kube-api-access-4q65s\" (UniqueName: \"kubernetes.io/projected/552a0155-ae0c-4811-b0ab-d11480e9cbd4-kube-api-access-4q65s\") pod \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\" (UID: \"552a0155-ae0c-4811-b0ab-d11480e9cbd4\") "
Oct 30 06:26:05.470705 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.470678    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/552a0155-ae0c-4811-b0ab-d11480e9cbd4-openshift-service-ca" pod "552a0155-ae0c-4811-b0ab-d11480e9cbd4" (UID: "552a0155-ae0c-4811-b0ab-d11480e9cbd4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:26:05.470782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.470762    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/552a0155-ae0c-4811-b0ab-d11480e9cbd4-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:05.473168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.473137    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-maas-api-tls" pod "552a0155-ae0c-4811-b0ab-d11480e9cbd4" (UID: "552a0155-ae0c-4811-b0ab-d11480e9cbd4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:26:05.473290 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.473193    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/552a0155-ae0c-4811-b0ab-d11480e9cbd4-kube-api-access-4q65s" pod "552a0155-ae0c-4811-b0ab-d11480e9cbd4" (UID: "552a0155-ae0c-4811-b0ab-d11480e9cbd4"). InnerVolumeSpecName "kube-api-access-4q65s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:26:05.473290 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.473233    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-metrics-certs" pod "552a0155-ae0c-4811-b0ab-d11480e9cbd4" (UID: "552a0155-ae0c-4811-b0ab-d11480e9cbd4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:26:05.572079 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.571982    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:05.572079 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.572013    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q65s\" (UniqueName: \"kubernetes.io/projected/552a0155-ae0c-4811-b0ab-d11480e9cbd4-kube-api-access-4q65s\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:05.572079 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.572029    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/552a0155-ae0c-4811-b0ab-d11480e9cbd4-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:05.915812 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:05.915780    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:06.142366 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:06.142331    5066 generic.go:356] "Generic (PLEG): container finished" podID="552a0155-ae0c-4811-b0ab-d11480e9cbd4" containerID="04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852" exitCode=0
Oct 30 06:26:06.142828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:06.142386    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"
Oct 30 06:26:06.142828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:06.142422    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m" event={"ID":"552a0155-ae0c-4811-b0ab-d11480e9cbd4","Type":"ContainerDied","Data":"04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852"}
Oct 30 06:26:06.142828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:06.142475    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m" event={"ID":"552a0155-ae0c-4811-b0ab-d11480e9cbd4","Type":"ContainerDied","Data":"782d3a325d62fa05c0f9e6eca4555dced890d606a3168f13e8fee1d18da53ad6"}
Oct 30 06:26:06.142828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:06.142497    5066 scope.go:122] "RemoveContainer" containerID="04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852"
Oct 30 06:26:06.152453 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:06.152434    5066 scope.go:122] "RemoveContainer" containerID="04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852"
Oct 30 06:26:06.152833 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:26:06.152803    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852\": container with ID starting with 04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852 not found: ID does not exist" containerID="04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852"
Oct 30 06:26:06.152942 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:06.152839    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852"} err="failed to get container status \"04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852\": rpc error: code = NotFound desc = could not find container \"04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852\": container with ID starting with 04be4dc0af5d8a252bf2875d54ec9cd87099a6642dfe0d4e36f41239f5e07852 not found: ID does not exist"
Oct 30 06:26:06.174128 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:06.174071    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:06.204580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:06.204548    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"]
Oct 30 06:26:06.216015 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:06.215991    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-25d881-54bb4fc58f-v5w6m"]
Oct 30 06:26:06.401486 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:06.401442    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="552a0155-ae0c-4811-b0ab-d11480e9cbd4" path="/var/lib/kubelet/pods/552a0155-ae0c-4811-b0ab-d11480e9cbd4/volumes"
Oct 30 06:26:06.618311 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:06.618228    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:08.136084 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:08.136043    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:10.046806 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:10.046770    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:12.011018 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:12.010963    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:13.380743 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:13.380709    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:13.555768 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:13.555729    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:13.730152 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:13.730101    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:13.900861 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:13.900828    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:15.715523 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:15.715491    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:17.651629 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:17.651536    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:19.512639 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:19.512604    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:21.442462 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:21.442431    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:23.151062 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:23.151023    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:24.553271 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.553240    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"]
Oct 30 06:26:24.553758 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.553443    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj" podUID="1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf" containerName="maas-api" containerID="cri-o://621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86" gracePeriod=30
Oct 30 06:26:24.681759 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.681727    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:24.824145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.824119    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:26:24.860846 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.860819    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-maas-api-tls\") pod \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") "
Oct 30 06:26:24.861031 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.860909    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-metrics-certs\") pod \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") "
Oct 30 06:26:24.861031 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.860932    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-openshift-service-ca\") pod \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") "
Oct 30 06:26:24.861031 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.860966    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-kube-api-access-9v9ht\" (UniqueName: \"kubernetes.io/projected/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-kube-api-access-9v9ht\") pod \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\" (UID: \"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf\") "
Oct 30 06:26:24.861385 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.861359    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-openshift-service-ca" pod "1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf" (UID: "1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:26:24.863009 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.862983    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:24.863420 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.863401    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-maas-api-tls" pod "1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf" (UID: "1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:26:24.863420 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.863412    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-kube-api-access-9v9ht" pod "1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf" (UID: "1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf"). InnerVolumeSpecName "kube-api-access-9v9ht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:26:24.863527 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.863422    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-metrics-certs" pod "1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf" (UID: "1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:26:24.962455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.962400    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-maas-api-tls\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:24.962455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.962426    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:24.962455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.962437    5066 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-openshift-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:24.962455 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:24.962446    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v9ht\" (UniqueName: \"kubernetes.io/projected/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf-kube-api-access-9v9ht\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:25.033401 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:25.033363    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:25.239306 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:25.239275    5066 generic.go:356] "Generic (PLEG): container finished" podID="1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf" containerID="621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86" exitCode=0
Oct 30 06:26:25.239534 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:25.239332    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"
Oct 30 06:26:25.239534 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:25.239363    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj" event={"ID":"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf","Type":"ContainerDied","Data":"621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86"}
Oct 30 06:26:25.239534 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:25.239422    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj" event={"ID":"1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf","Type":"ContainerDied","Data":"18348fc529cdbaa83d7d3a83573108fd3504c098ea8d150b1c9e70ca382e8612"}
Oct 30 06:26:25.239534 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:25.239446    5066 scope.go:122] "RemoveContainer" containerID="621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86"
Oct 30 06:26:25.252828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:25.252668    5066 scope.go:122] "RemoveContainer" containerID="621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86"
Oct 30 06:26:25.253287 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:26:25.253264    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86\": container with ID starting with 621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86 not found: ID does not exist" containerID="621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86"
Oct 30 06:26:25.253398 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:25.253294    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86"} err="failed to get container status \"621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86\": rpc error: code = NotFound desc = could not find container \"621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86\": container with ID starting with 621b3f7f00222b2ed867cfd52baab8b5cc03e101b3e8d1761fd8cc9c9ecbcb86 not found: ID does not exist"
Oct 30 06:26:25.299197 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:25.299170    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"]
Oct 30 06:26:25.313335 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:25.313306    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f6302c-8657c48c94-m2whj"]
Oct 30 06:26:25.366195 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:25.365765    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:25.999543 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:25.999515    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:26.227696 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:26.227634    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:26.402989 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:26.402910    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf" path="/var/lib/kubelet/pods/1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf/volumes"
Oct 30 06:26:26.656485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:26.656454    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:27.533081 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:27.533047    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:29.353644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:29.353610    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:31.062247 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:31.062215    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:32.552469 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:26:32.552435    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:26:32.861285 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:32.861207    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:33.750822 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:33.750793    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:33.922777 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:33.922749    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:34.119067 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:34.118977    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:34.294183 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:34.294155    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:34.628756 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:34.628720    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:34.788067 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:34.788037    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:35.051300 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:35.051270    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:36.815309 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:36.815275    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:36.948799 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:36.948774    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:26:36.967917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:36.967892    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:26:38.617737 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:38.617706    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:39.541548 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:39.541512    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"]
Oct 30 06:26:39.574199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:39.574165    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"]
Oct 30 06:26:39.782802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:39.782770    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:40.508564 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:40.508529    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:42.250019 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:42.249988    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:43.972115 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:43.972083    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:45.341187 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.341157    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:45.508152 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508116    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"]
Oct 30 06:26:45.508542 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508530    5066 cpu_manager.go:414] "containerMap: removing container" podUID="05d2bec0-3663-4c53-8b93-b0bb50191185" containerName="istio-proxy"
Oct 30 06:26:45.508621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508544    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05d2bec0-3663-4c53-8b93-b0bb50191185" containerName="istio-proxy"
Oct 30 06:26:45.508621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508563    5066 cpu_manager.go:414] "containerMap: removing container" podUID="dafc9b85-538c-4f66-8578-012bad182b0a" containerName="istio-proxy"
Oct 30 06:26:45.508621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508568    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dafc9b85-538c-4f66-8578-012bad182b0a" containerName="istio-proxy"
Oct 30 06:26:45.508621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508579    5066 cpu_manager.go:414] "containerMap: removing container" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main"
Oct 30 06:26:45.508621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508598    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main"
Oct 30 06:26:45.508621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508614    5066 cpu_manager.go:414] "containerMap: removing container" podUID="552a0155-ae0c-4811-b0ab-d11480e9cbd4" containerName="maas-api"
Oct 30 06:26:45.508621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508618    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="552a0155-ae0c-4811-b0ab-d11480e9cbd4" containerName="maas-api"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508625    5066 cpu_manager.go:414] "containerMap: removing container" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508631    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508640    5066 cpu_manager.go:414] "containerMap: removing container" podUID="32a02a07-cb03-4262-b912-27a8b26b7c50" containerName="revoke-keys"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508645    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32a02a07-cb03-4262-b912-27a8b26b7c50" containerName="revoke-keys"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508658    5066 cpu_manager.go:414] "containerMap: removing container" podUID="1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf" containerName="maas-api"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508663    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf" containerName="maas-api"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508726    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="dafc9b85-538c-4f66-8578-012bad182b0a" containerName="istio-proxy"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508734    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef3506e3-2464-46ed-8808-3c5d681330ad" containerName="main"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508743    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="552a0155-ae0c-4811-b0ab-d11480e9cbd4" containerName="maas-api"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508749    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="8bdc1785-b6ce-4de9-bda2-6127f1964b13" containerName="main"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508755    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="32a02a07-cb03-4262-b912-27a8b26b7c50" containerName="revoke-keys"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508761    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f9c9408-f6b3-4b64-8238-4d8dcbd11dbf" containerName="maas-api"
Oct 30 06:26:45.508877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.508768    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="05d2bec0-3663-4c53-8b93-b0bb50191185" containerName="istio-proxy"
Oct 30 06:26:45.513401 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.513372    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:45.520630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.516366    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:45.538845 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.538816    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"]
Oct 30 06:26:45.610834 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.610758    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 06:26:45.610834 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.610758    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 06:26:45.611753 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.611740    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 06:26:45.612052 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.612041    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:26:45.668324 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.668286    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hhg2\" (UniqueName: \"kubernetes.io/projected/b80e39d8-daad-4c50-aabb-458e5505dcde-kube-api-access-6hhg2\") pod \"maas-api-revoke-sub-c3562a41-cq24s\" (UID: \"b80e39d8-daad-4c50-aabb-458e5505dcde\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:45.668503 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.668408    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c3562a41-cq24s\" (UID: \"b80e39d8-daad-4c50-aabb-458e5505dcde\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:45.668503 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.668462    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c3562a41-cq24s\" (UID: \"b80e39d8-daad-4c50-aabb-458e5505dcde\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:45.769875 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.769839    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hhg2\" (UniqueName: \"kubernetes.io/projected/b80e39d8-daad-4c50-aabb-458e5505dcde-kube-api-access-6hhg2\") pod \"maas-api-revoke-sub-c3562a41-cq24s\" (UID: \"b80e39d8-daad-4c50-aabb-458e5505dcde\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:45.770073 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.769922    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c3562a41-cq24s\" (UID: \"b80e39d8-daad-4c50-aabb-458e5505dcde\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:45.770073 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.769958    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c3562a41-cq24s\" (UID: \"b80e39d8-daad-4c50-aabb-458e5505dcde\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:45.770659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.770623    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c3562a41-cq24s\" (UID: \"b80e39d8-daad-4c50-aabb-458e5505dcde\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:45.772494 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.772476    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c3562a41-cq24s\" (UID: \"b80e39d8-daad-4c50-aabb-458e5505dcde\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:45.795985 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.795951    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hhg2\" (UniqueName: \"kubernetes.io/projected/b80e39d8-daad-4c50-aabb-458e5505dcde-kube-api-access-6hhg2\") pod \"maas-api-revoke-sub-c3562a41-cq24s\" (UID: \"b80e39d8-daad-4c50-aabb-458e5505dcde\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:45.828838 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:45.828805    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:46.140404 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:46.140379    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"]
Oct 30 06:26:46.142023 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:26:46.141996    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb80e39d8_daad_4c50_aabb_458e5505dcde.slice/crio-334111f1910276f6e1a8acbd3f1572a0bf9b1fa95dceed97b00dfa850547f45a WatchSource:0}: Error finding container 334111f1910276f6e1a8acbd3f1572a0bf9b1fa95dceed97b00dfa850547f45a: Status 404 returned error can't find the container with id 334111f1910276f6e1a8acbd3f1572a0bf9b1fa95dceed97b00dfa850547f45a
Oct 30 06:26:46.355709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:46.355676    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s" event={"ID":"b80e39d8-daad-4c50-aabb-458e5505dcde","Type":"ContainerStarted","Data":"334111f1910276f6e1a8acbd3f1572a0bf9b1fa95dceed97b00dfa850547f45a"}
Oct 30 06:26:47.369702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:47.369611    5066 generic.go:356] "Generic (PLEG): container finished" podID="b80e39d8-daad-4c50-aabb-458e5505dcde" containerID="b1d1715fccd9251b1912cd5c1e97b915ddd4b69544c9f56b2814f4452b29459e" exitCode=0
Oct 30 06:26:47.370087 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:47.369698    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s" event={"ID":"b80e39d8-daad-4c50-aabb-458e5505dcde","Type":"ContainerDied","Data":"b1d1715fccd9251b1912cd5c1e97b915ddd4b69544c9f56b2814f4452b29459e"}
Oct 30 06:26:48.516715 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:48.516691    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:48.598578 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:48.598546    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b80e39d8-daad-4c50-aabb-458e5505dcde-kube-api-access-6hhg2\" (UniqueName: \"kubernetes.io/projected/b80e39d8-daad-4c50-aabb-458e5505dcde-kube-api-access-6hhg2\") pod \"b80e39d8-daad-4c50-aabb-458e5505dcde\" (UID: \"b80e39d8-daad-4c50-aabb-458e5505dcde\") "
Oct 30 06:26:48.598772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:48.598656    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-openshift-ca\") pod \"b80e39d8-daad-4c50-aabb-458e5505dcde\" (UID: \"b80e39d8-daad-4c50-aabb-458e5505dcde\") "
Oct 30 06:26:48.598772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:48.598758    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-xks-ca\") pod \"b80e39d8-daad-4c50-aabb-458e5505dcde\" (UID: \"b80e39d8-daad-4c50-aabb-458e5505dcde\") "
Oct 30 06:26:48.599019 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:48.598992    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-openshift-ca" pod "b80e39d8-daad-4c50-aabb-458e5505dcde" (UID: "b80e39d8-daad-4c50-aabb-458e5505dcde"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:26:48.599120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:48.599100    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:48.600993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:48.600967    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b80e39d8-daad-4c50-aabb-458e5505dcde-kube-api-access-6hhg2" pod "b80e39d8-daad-4c50-aabb-458e5505dcde" (UID: "b80e39d8-daad-4c50-aabb-458e5505dcde"). InnerVolumeSpecName "kube-api-access-6hhg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:26:48.601127 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:48.601024    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-xks-ca" pod "b80e39d8-daad-4c50-aabb-458e5505dcde" (UID: "b80e39d8-daad-4c50-aabb-458e5505dcde"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:26:48.700718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:48.700675    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b80e39d8-daad-4c50-aabb-458e5505dcde-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:48.700718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:48.700706    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hhg2\" (UniqueName: \"kubernetes.io/projected/b80e39d8-daad-4c50-aabb-458e5505dcde-kube-api-access-6hhg2\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:49.379580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:49.379552    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"
Oct 30 06:26:49.379777 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:49.379608    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s" event={"ID":"b80e39d8-daad-4c50-aabb-458e5505dcde","Type":"ContainerDied","Data":"334111f1910276f6e1a8acbd3f1572a0bf9b1fa95dceed97b00dfa850547f45a"}
Oct 30 06:26:49.379777 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:49.379660    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="334111f1910276f6e1a8acbd3f1572a0bf9b1fa95dceed97b00dfa850547f45a"
Oct 30 06:26:51.379115 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:51.379071    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:53.691217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:53.691178    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:54.562222 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.562165    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" containerID="cri-o://e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5" gracePeriod=45
Oct 30 06:26:54.594119 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.594088    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main" containerID="cri-o://9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e" gracePeriod=45
Oct 30 06:26:54.844745 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.844723    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 06:26:54.901261 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.901224    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 06:26:54.967203 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.967170    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/17de89f6-37da-48fb-97cc-f982d94e83a7-tls-certs\" (UniqueName: \"kubernetes.io/secret/17de89f6-37da-48fb-97cc-f982d94e83a7-tls-certs\") pod \"17de89f6-37da-48fb-97cc-f982d94e83a7\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") "
Oct 30 06:26:54.967400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.967220    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-dshm\") pod \"17de89f6-37da-48fb-97cc-f982d94e83a7\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") "
Oct 30 06:26:54.967400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.967310    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-home\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-home\") pod \"17de89f6-37da-48fb-97cc-f982d94e83a7\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") "
Oct 30 06:26:54.967400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.967351    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17de89f6-37da-48fb-97cc-f982d94e83a7-kube-api-access-k4rwc\" (UniqueName: \"kubernetes.io/projected/17de89f6-37da-48fb-97cc-f982d94e83a7-kube-api-access-k4rwc\") pod \"17de89f6-37da-48fb-97cc-f982d94e83a7\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") "
Oct 30 06:26:54.967400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.967392    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-tmp-dir\") pod \"17de89f6-37da-48fb-97cc-f982d94e83a7\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") "
Oct 30 06:26:54.967668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.967434    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-model-cache\") pod \"17de89f6-37da-48fb-97cc-f982d94e83a7\" (UID: \"17de89f6-37da-48fb-97cc-f982d94e83a7\") "
Oct 30 06:26:54.967947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.967922    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-model-cache" pod "17de89f6-37da-48fb-97cc-f982d94e83a7" (UID: "17de89f6-37da-48fb-97cc-f982d94e83a7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:54.967947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.967935    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-tmp-dir" pod "17de89f6-37da-48fb-97cc-f982d94e83a7" (UID: "17de89f6-37da-48fb-97cc-f982d94e83a7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:54.968171 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.968053    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-home" pod "17de89f6-37da-48fb-97cc-f982d94e83a7" (UID: "17de89f6-37da-48fb-97cc-f982d94e83a7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:54.969818 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.969790    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17de89f6-37da-48fb-97cc-f982d94e83a7-tls-certs" pod "17de89f6-37da-48fb-97cc-f982d94e83a7" (UID: "17de89f6-37da-48fb-97cc-f982d94e83a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:26:54.970010 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.969985    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17de89f6-37da-48fb-97cc-f982d94e83a7-kube-api-access-k4rwc" pod "17de89f6-37da-48fb-97cc-f982d94e83a7" (UID: "17de89f6-37da-48fb-97cc-f982d94e83a7"). InnerVolumeSpecName "kube-api-access-k4rwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:26:54.970548 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:54.970533    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-dshm" pod "17de89f6-37da-48fb-97cc-f982d94e83a7" (UID: "17de89f6-37da-48fb-97cc-f982d94e83a7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:55.068543 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.068509    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e98db60e-37c7-45cd-80d1-8770e15aaa0b-kube-api-access-zkc29\" (UniqueName: \"kubernetes.io/projected/e98db60e-37c7-45cd-80d1-8770e15aaa0b-kube-api-access-zkc29\") pod \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") "
Oct 30 06:26:55.068774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.068682    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-home\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-home\") pod \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") "
Oct 30 06:26:55.068774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.068733    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-model-cache\") pod \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") "
Oct 30 06:26:55.068774 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.068764    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tls-certs\" (UniqueName: \"kubernetes.io/secret/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tls-certs\") pod \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") "
Oct 30 06:26:55.068955 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.068797    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-dshm\") pod \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") "
Oct 30 06:26:55.068955 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.068833    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tmp-dir\") pod \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\" (UID: \"e98db60e-37c7-45cd-80d1-8770e15aaa0b\") "
Oct 30 06:26:55.069066 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.068940    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-home" pod "e98db60e-37c7-45cd-80d1-8770e15aaa0b" (UID: "e98db60e-37c7-45cd-80d1-8770e15aaa0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:55.069129 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.069042    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-model-cache" pod "e98db60e-37c7-45cd-80d1-8770e15aaa0b" (UID: "e98db60e-37c7-45cd-80d1-8770e15aaa0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:55.069129 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.069094    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tmp-dir" pod "e98db60e-37c7-45cd-80d1-8770e15aaa0b" (UID: "e98db60e-37c7-45cd-80d1-8770e15aaa0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:55.069331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.069247    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:55.069331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.069270    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:55.069331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.069279    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:55.069331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.069288    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:55.069331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.069301    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4rwc\" (UniqueName: \"kubernetes.io/projected/17de89f6-37da-48fb-97cc-f982d94e83a7-kube-api-access-k4rwc\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:55.069331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.069313    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:55.069331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.069327    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:55.069331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.069338    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17de89f6-37da-48fb-97cc-f982d94e83a7-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:55.069331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.069345    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17de89f6-37da-48fb-97cc-f982d94e83a7-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:55.071040 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.070986    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-dshm" pod "e98db60e-37c7-45cd-80d1-8770e15aaa0b" (UID: "e98db60e-37c7-45cd-80d1-8770e15aaa0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:26:55.071040 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.071002    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e98db60e-37c7-45cd-80d1-8770e15aaa0b-kube-api-access-zkc29" pod "e98db60e-37c7-45cd-80d1-8770e15aaa0b" (UID: "e98db60e-37c7-45cd-80d1-8770e15aaa0b"). InnerVolumeSpecName "kube-api-access-zkc29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:26:55.071040 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.070984    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tls-certs" pod "e98db60e-37c7-45cd-80d1-8770e15aaa0b" (UID: "e98db60e-37c7-45cd-80d1-8770e15aaa0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:26:55.169910 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.169872    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkc29\" (UniqueName: \"kubernetes.io/projected/e98db60e-37c7-45cd-80d1-8770e15aaa0b-kube-api-access-zkc29\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:55.169910 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.169903    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e98db60e-37c7-45cd-80d1-8770e15aaa0b-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:55.169910 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.169914    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e98db60e-37c7-45cd-80d1-8770e15aaa0b-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:26:55.409157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.409125    5066 generic.go:356] "Generic (PLEG): container finished" podID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerID="e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5" exitCode=0
Oct 30 06:26:55.409349 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.409170    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"
Oct 30 06:26:55.409349 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.409212    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" event={"ID":"17de89f6-37da-48fb-97cc-f982d94e83a7","Type":"ContainerDied","Data":"e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5"}
Oct 30 06:26:55.409349 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.409256    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" event={"ID":"17de89f6-37da-48fb-97cc-f982d94e83a7","Type":"ContainerDied","Data":"6fc8bc4f70e3c6c71df251ff220e96e8403c90649f840e43886a68ce8d971e16"}
Oct 30 06:26:55.409349 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.409279    5066 scope.go:122] "RemoveContainer" containerID="e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5"
Oct 30 06:26:55.410672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.410650    5066 generic.go:356] "Generic (PLEG): container finished" podID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerID="9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e" exitCode=0
Oct 30 06:26:55.410796 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.410683    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" event={"ID":"e98db60e-37c7-45cd-80d1-8770e15aaa0b","Type":"ContainerDied","Data":"9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e"}
Oct 30 06:26:55.410796 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.410706    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4" event={"ID":"e98db60e-37c7-45cd-80d1-8770e15aaa0b","Type":"ContainerDied","Data":"568a63422b5ca298429c68db4ce66867452fbe8b4043a5702662154d2236a8d2"}
Oct 30 06:26:55.410796 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.410714    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"
Oct 30 06:26:55.418880 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.418863    5066 scope.go:122] "RemoveContainer" containerID="e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5"
Oct 30 06:26:55.419201 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:26:55.419180    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5\": container with ID starting with e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5 not found: ID does not exist" containerID="e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5"
Oct 30 06:26:55.419295 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.419212    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5"} err="failed to get container status \"e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5\": rpc error: code = NotFound desc = could not find container \"e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5\": container with ID starting with e87fe607df5012ac278f04ba7594d87960f26bd78ae11dcec68f9292f5fbe0f5 not found: ID does not exist"
Oct 30 06:26:55.419295 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.419234    5066 scope.go:122] "RemoveContainer" containerID="9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e"
Oct 30 06:26:55.427838 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.427818    5066 scope.go:122] "RemoveContainer" containerID="9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e"
Oct 30 06:26:55.428093 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:26:55.428076    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e\": container with ID starting with 9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e not found: ID does not exist" containerID="9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e"
Oct 30 06:26:55.428168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.428104    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e"} err="failed to get container status \"9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e\": rpc error: code = NotFound desc = could not find container \"9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e\": container with ID starting with 9cb955c22a95f4508f5cc83bc13f6785f813d93bc0d2d1402a47466c2d9fa15e not found: ID does not exist"
Oct 30 06:26:55.467813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.467780    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"]
Oct 30 06:26:55.481323 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.481295    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p"]
Oct 30 06:26:55.520998 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.520967    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"]
Oct 30 06:26:55.527542 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.527514    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-c78066/simulator-w2-c78066-kserve-66d4f5f87f-fc4n4"]
Oct 30 06:26:55.794956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:55.794860    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c78066/distinct-w2-c78066-kserve-5b47cd8657-z7d6p" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 06:26:56.401037 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:56.401005    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" path="/var/lib/kubelet/pods/17de89f6-37da-48fb-97cc-f982d94e83a7/volumes"
Oct 30 06:26:56.401450 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:56.401404    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" path="/var/lib/kubelet/pods/e98db60e-37c7-45cd-80d1-8770e15aaa0b/volumes"
Oct 30 06:26:58.807068 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:58.807040    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:26:58.991525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:26:58.991491    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:00.626060 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:00.626012    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:00.789614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:00.789573    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:01.278580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:01.278545    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:01.447176 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:01.447135    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:01.953250 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:01.953209    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:02.552516 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:27:02.552481    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:27:06.404305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:06.404278    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:06.585776 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:06.585742    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:08.464240 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:08.464203    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:08.654450 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:08.654413    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:08.678543 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:08.675809    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"]
Oct 30 06:27:08.678543 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:08.676222    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" podUID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" containerName="istio-proxy" containerID="cri-o://8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672" gracePeriod=30
Oct 30 06:27:08.822069 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:08.821983    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:13.860709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:13.860675    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5974d5d96-flgp2"]
Oct 30 06:27:13.861068 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:13.860877    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-5974d5d96-flgp2" podUID="1b95079a-128b-45eb-be31-d240aeccea74" containerName="manager" containerID="cri-o://131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654" gracePeriod=30
Oct 30 06:27:13.988452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:13.988419    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 06:27:14.046209 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046180    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-podinfo\") pod \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") "
Oct 30 06:27:14.046397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046222    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-socket\") pod \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") "
Oct 30 06:27:14.046397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046252    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-credential-socket\") pod \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") "
Oct 30 06:27:14.046397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046294    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-kube-api-access-9mmjk\" (UniqueName: \"kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-kube-api-access-9mmjk\") pod \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") "
Oct 30 06:27:14.046397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046340    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-token\" (UniqueName: \"kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-token\") pod \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") "
Oct 30 06:27:14.046397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046371    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-data\") pod \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") "
Oct 30 06:27:14.047820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046416    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-certs\") pod \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") "
Oct 30 06:27:14.047820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046441    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istiod-ca-cert\") pod \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") "
Oct 30 06:27:14.047820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046508    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-envoy\") pod \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\" (UID: \"bf2d37a5-3086-4ac9-892f-06ab233f1e70\") "
Oct 30 06:27:14.047820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046567    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-socket" pod "bf2d37a5-3086-4ac9-892f-06ab233f1e70" (UID: "bf2d37a5-3086-4ac9-892f-06ab233f1e70"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:27:14.047820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046583    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-credential-socket" pod "bf2d37a5-3086-4ac9-892f-06ab233f1e70" (UID: "bf2d37a5-3086-4ac9-892f-06ab233f1e70"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:27:14.047820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046811    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-data" pod "bf2d37a5-3086-4ac9-892f-06ab233f1e70" (UID: "bf2d37a5-3086-4ac9-892f-06ab233f1e70"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:27:14.047820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046881    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-certs" pod "bf2d37a5-3086-4ac9-892f-06ab233f1e70" (UID: "bf2d37a5-3086-4ac9-892f-06ab233f1e70"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:27:14.047820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046891    5066 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:27:14.047820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.046953    5066 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-credential-socket\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:27:14.047820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.047078    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istiod-ca-cert" pod "bf2d37a5-3086-4ac9-892f-06ab233f1e70" (UID: "bf2d37a5-3086-4ac9-892f-06ab233f1e70"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:27:14.049604 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.049560    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-token" pod "bf2d37a5-3086-4ac9-892f-06ab233f1e70" (UID: "bf2d37a5-3086-4ac9-892f-06ab233f1e70"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:27:14.050710 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.050686    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-kube-api-access-9mmjk" pod "bf2d37a5-3086-4ac9-892f-06ab233f1e70" (UID: "bf2d37a5-3086-4ac9-892f-06ab233f1e70"). InnerVolumeSpecName "kube-api-access-9mmjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:27:14.050803 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.050774    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-envoy" pod "bf2d37a5-3086-4ac9-892f-06ab233f1e70" (UID: "bf2d37a5-3086-4ac9-892f-06ab233f1e70"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:27:14.051012 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.050988    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-podinfo" pod "bf2d37a5-3086-4ac9-892f-06ab233f1e70" (UID: "bf2d37a5-3086-4ac9-892f-06ab233f1e70"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 06:27:14.101246 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.101220    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 06:27:14.148084 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.148042    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-metrics-certs\") pod \"1b95079a-128b-45eb-be31-d240aeccea74\" (UID: \"1b95079a-128b-45eb-be31-d240aeccea74\") "
Oct 30 06:27:14.148300 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.148168    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-webhook-cert\") pod \"1b95079a-128b-45eb-be31-d240aeccea74\" (UID: \"1b95079a-128b-45eb-be31-d240aeccea74\") "
Oct 30 06:27:14.148300 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.148277    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b95079a-128b-45eb-be31-d240aeccea74-kube-api-access-ksxt2\" (UniqueName: \"kubernetes.io/projected/1b95079a-128b-45eb-be31-d240aeccea74-kube-api-access-ksxt2\") pod \"1b95079a-128b-45eb-be31-d240aeccea74\" (UID: \"1b95079a-128b-45eb-be31-d240aeccea74\") "
Oct 30 06:27:14.148933 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.148880    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mmjk\" (UniqueName: \"kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-kube-api-access-9mmjk\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:27:14.149060 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.148965    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:27:14.149060 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.148980    5066 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-data\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:27:14.149060 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.148992    5066 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-workload-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:27:14.149060 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.149004    5066 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istiod-ca-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:27:14.149060 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.149016    5066 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-envoy\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:27:14.149060 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.149028    5066 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf2d37a5-3086-4ac9-892f-06ab233f1e70-istio-podinfo\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:27:14.151026 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.150999    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-webhook-cert" pod "1b95079a-128b-45eb-be31-d240aeccea74" (UID: "1b95079a-128b-45eb-be31-d240aeccea74"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:27:14.151242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.151138    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b95079a-128b-45eb-be31-d240aeccea74-kube-api-access-ksxt2" pod "1b95079a-128b-45eb-be31-d240aeccea74" (UID: "1b95079a-128b-45eb-be31-d240aeccea74"). InnerVolumeSpecName "kube-api-access-ksxt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:27:14.151362 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.151300    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-metrics-certs" pod "1b95079a-128b-45eb-be31-d240aeccea74" (UID: "1b95079a-128b-45eb-be31-d240aeccea74"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:27:14.250449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.250369    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksxt2\" (UniqueName: \"kubernetes.io/projected/1b95079a-128b-45eb-be31-d240aeccea74-kube-api-access-ksxt2\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:27:14.250449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.250399    5066 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-metrics-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:27:14.250449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.250408    5066 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b95079a-128b-45eb-be31-d240aeccea74-webhook-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:27:14.510671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.510556    5066 generic.go:356] "Generic (PLEG): container finished" podID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" containerID="8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672" exitCode=0
Oct 30 06:27:14.510671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.510640    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"
Oct 30 06:27:14.510671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.510650    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" event={"ID":"bf2d37a5-3086-4ac9-892f-06ab233f1e70","Type":"ContainerDied","Data":"8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672"}
Oct 30 06:27:14.510956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.510715    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t" event={"ID":"bf2d37a5-3086-4ac9-892f-06ab233f1e70","Type":"ContainerDied","Data":"b61fd416d8c3215e0eb2a11a983c1a196fad2572008fee9cedaed0447563af45"}
Oct 30 06:27:14.510956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.510729    5066 scope.go:122] "RemoveContainer" containerID="8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672"
Oct 30 06:27:14.512632 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.512604    5066 generic.go:356] "Generic (PLEG): container finished" podID="1b95079a-128b-45eb-be31-d240aeccea74" containerID="131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654" exitCode=0
Oct 30 06:27:14.512761 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.512660    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5974d5d96-flgp2"
Oct 30 06:27:14.512761 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.512713    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5974d5d96-flgp2" event={"ID":"1b95079a-128b-45eb-be31-d240aeccea74","Type":"ContainerDied","Data":"131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654"}
Oct 30 06:27:14.512761 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.512751    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5974d5d96-flgp2" event={"ID":"1b95079a-128b-45eb-be31-d240aeccea74","Type":"ContainerDied","Data":"b2be97de7ca56ad17d49db13180bab68c1905e80f12f7c4b11d5d0de0023845a"}
Oct 30 06:27:14.520374 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.520359    5066 scope.go:122] "RemoveContainer" containerID="8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672"
Oct 30 06:27:14.520737 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:27:14.520717    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672\": container with ID starting with 8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672 not found: ID does not exist" containerID="8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672"
Oct 30 06:27:14.520789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.520746    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672"} err="failed to get container status \"8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672\": rpc error: code = NotFound desc = could not find container \"8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672\": container with ID starting with 8f9405cd4e3da03011455be899b8b940ec704096c00737af6c3966a76754f672 not found: ID does not exist"
Oct 30 06:27:14.520789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.520764    5066 scope.go:122] "RemoveContainer" containerID="131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654"
Oct 30 06:27:14.530474 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.530456    5066 scope.go:122] "RemoveContainer" containerID="131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654"
Oct 30 06:27:14.530783 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:27:14.530762    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654\": container with ID starting with 131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654 not found: ID does not exist" containerID="131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654"
Oct 30 06:27:14.530853 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.530790    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654"} err="failed to get container status \"131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654\": rpc error: code = NotFound desc = could not find container \"131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654\": container with ID starting with 131e97310bf18dc0741881a6edbbcd8d4efa159f1021e668be493ef6b41ef654 not found: ID does not exist"
Oct 30 06:27:14.569767 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.569739    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"]
Oct 30 06:27:14.584168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.584142    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c78066-openshift-default-5696c477f7-mlt2t"]
Oct 30 06:27:14.616282 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.616246    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5974d5d96-flgp2"]
Oct 30 06:27:14.619996 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:14.619968    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5974d5d96-flgp2"]
Oct 30 06:27:16.400230 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:16.400203    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b95079a-128b-45eb-be31-d240aeccea74" path="/var/lib/kubelet/pods/1b95079a-128b-45eb-be31-d240aeccea74/volumes"
Oct 30 06:27:16.400725 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:16.400709    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" path="/var/lib/kubelet/pods/bf2d37a5-3086-4ac9-892f-06ab233f1e70/volumes"
Oct 30 06:27:24.348539 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.348501    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"]
Oct 30 06:27:24.349016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.348964    5066 cpu_manager.go:414] "containerMap: removing container" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main"
Oct 30 06:27:24.349016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.348977    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main"
Oct 30 06:27:24.349016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.348988    5066 cpu_manager.go:414] "containerMap: removing container" podUID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" containerName="istio-proxy"
Oct 30 06:27:24.349016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.348993    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" containerName="istio-proxy"
Oct 30 06:27:24.349016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.349002    5066 cpu_manager.go:414] "containerMap: removing container" podUID="1b95079a-128b-45eb-be31-d240aeccea74" containerName="manager"
Oct 30 06:27:24.349016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.349007    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b95079a-128b-45eb-be31-d240aeccea74" containerName="manager"
Oct 30 06:27:24.349016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.349013    5066 cpu_manager.go:414] "containerMap: removing container" podUID="b80e39d8-daad-4c50-aabb-458e5505dcde" containerName="revoke-keys"
Oct 30 06:27:24.349016 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.349017    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b80e39d8-daad-4c50-aabb-458e5505dcde" containerName="revoke-keys"
Oct 30 06:27:24.349361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.349036    5066 cpu_manager.go:414] "containerMap: removing container" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main"
Oct 30 06:27:24.349361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.349043    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main"
Oct 30 06:27:24.349361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.349177    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="17de89f6-37da-48fb-97cc-f982d94e83a7" containerName="main"
Oct 30 06:27:24.349361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.349189    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf2d37a5-3086-4ac9-892f-06ab233f1e70" containerName="istio-proxy"
Oct 30 06:27:24.349361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.349199    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b95079a-128b-45eb-be31-d240aeccea74" containerName="manager"
Oct 30 06:27:24.349361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.349207    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="b80e39d8-daad-4c50-aabb-458e5505dcde" containerName="revoke-keys"
Oct 30 06:27:24.349361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.349217    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="e98db60e-37c7-45cd-80d1-8770e15aaa0b" containerName="main"
Oct 30 06:27:24.353789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.353771    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.368673 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.368649    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"]
Oct 30 06:27:24.372132 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.372114    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.381365 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.381340    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ntk7q\""
Oct 30 06:27:24.383792 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.383768    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"]
Oct 30 06:27:24.410527 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.410495    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"]
Oct 30 06:27:24.447611 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.447548    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.447611 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.447614    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.447911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.447688    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bg5z\" (UniqueName: \"kubernetes.io/projected/74763aed-8beb-4048-8d84-7e52fa1c19dd-kube-api-access-6bg5z\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.447911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.447725    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74763aed-8beb-4048-8d84-7e52fa1c19dd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.447911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.447745    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.447911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.447803    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.447911 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.447894    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.448181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.447931    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k5pg\" (UniqueName: \"kubernetes.io/projected/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-kube-api-access-9k5pg\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.448181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.448001    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.448181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.448030    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.448181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.448052    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.448181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.448075    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.448181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.448112    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.448181 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.448156    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.448445 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.448221    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74763aed-8beb-4048-8d84-7e52fa1c19dd-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.448445 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.448345    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74763aed-8beb-4048-8d84-7e52fa1c19dd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.448445 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.448371    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.448445 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.448400    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.549094 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549058    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.549094 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549096    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.549428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549121    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.549428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549138    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bg5z\" (UniqueName: \"kubernetes.io/projected/74763aed-8beb-4048-8d84-7e52fa1c19dd-kube-api-access-6bg5z\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.549428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549172    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74763aed-8beb-4048-8d84-7e52fa1c19dd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.549428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549199    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.549428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549233    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.549428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549266    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.549428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549295    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k5pg\" (UniqueName: \"kubernetes.io/projected/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-kube-api-access-9k5pg\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.549428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549327    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.549428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549357    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.549428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549382    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.549428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549407    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.550061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549432    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.550061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549460    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.550061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549493    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74763aed-8beb-4048-8d84-7e52fa1c19dd-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.550061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549623    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74763aed-8beb-4048-8d84-7e52fa1c19dd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.550061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549630    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.550061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549641    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.550061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549828    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.550061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549902    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.550061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549916    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.550061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.550004    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.550061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.549653    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.550761 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.550165    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.550761 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.550317    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.550761 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.550622    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74763aed-8beb-4048-8d84-7e52fa1c19dd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.551098 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.550760    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.551771 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.551750    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74763aed-8beb-4048-8d84-7e52fa1c19dd-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.552262 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.552247    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.552262 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.552256    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.552901 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.552878    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74763aed-8beb-4048-8d84-7e52fa1c19dd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.574562 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.574535    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74763aed-8beb-4048-8d84-7e52fa1c19dd-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.574716 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.574570    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bg5z\" (UniqueName: \"kubernetes.io/projected/74763aed-8beb-4048-8d84-7e52fa1c19dd-kube-api-access-6bg5z\") pod \"maas-default-gateway-openshift-default-58847f9fc5-n57mc\" (UID: \"74763aed-8beb-4048-8d84-7e52fa1c19dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.574785 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.574760    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.575247 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.575224    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k5pg\" (UniqueName: \"kubernetes.io/projected/4d54096d-00ef-4c39-a0e8-6d1abf5f6640-kube-api-access-9k5pg\") pod \"maas-default-gateway-openshift-default-58847f9fc5-qgv7r\" (UID: \"4d54096d-00ef-4c39-a0e8-6d1abf5f6640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:24.668061 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.668038    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:24.687401 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:24.687378    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:25.001882 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.001848    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"]
Oct 30 06:27:25.003202 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:27:25.003176    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74763aed_8beb_4048_8d84_7e52fa1c19dd.slice/crio-0217709df679b58f2acf2bb324edf6f2d54371f94a19fd8a6da21461cc0d318d WatchSource:0}: Error finding container 0217709df679b58f2acf2bb324edf6f2d54371f94a19fd8a6da21461cc0d318d: Status 404 returned error can't find the container with id 0217709df679b58f2acf2bb324edf6f2d54371f94a19fd8a6da21461cc0d318d
Oct 30 06:27:25.005232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.005210    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"]
Oct 30 06:27:25.005878 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.005829    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:27:25.005919 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.005885    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:27:25.005919 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.005912    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:27:25.007979 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:27:25.007954    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d54096d_00ef_4c39_a0e8_6d1abf5f6640.slice/crio-8680c1f664b3a728e62e66ac202776852172c5af6c04e7ab3b4b8cfe7bd6adf8 WatchSource:0}: Error finding container 8680c1f664b3a728e62e66ac202776852172c5af6c04e7ab3b4b8cfe7bd6adf8: Status 404 returned error can't find the container with id 8680c1f664b3a728e62e66ac202776852172c5af6c04e7ab3b4b8cfe7bd6adf8
Oct 30 06:27:25.013302 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.013271    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:27:25.013393 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.013332    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:27:25.013393 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.013360    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:27:25.582705 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.582411    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r" event={"ID":"4d54096d-00ef-4c39-a0e8-6d1abf5f6640","Type":"ContainerStarted","Data":"8fd4bf7f2bccdb2630643a2cc04e3d678bf29553ba289b978d1b77a8507aa30f"}
Oct 30 06:27:25.583104 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.582739    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r" event={"ID":"4d54096d-00ef-4c39-a0e8-6d1abf5f6640","Type":"ContainerStarted","Data":"8680c1f664b3a728e62e66ac202776852172c5af6c04e7ab3b4b8cfe7bd6adf8"}
Oct 30 06:27:25.584105 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.584081    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc" event={"ID":"74763aed-8beb-4048-8d84-7e52fa1c19dd","Type":"ContainerStarted","Data":"8c175e0311c1ed91ef84f02b0959218a5b1a91bf07029ef70908b9ae356ed6a7"}
Oct 30 06:27:25.584221 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.584147    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc" event={"ID":"74763aed-8beb-4048-8d84-7e52fa1c19dd","Type":"ContainerStarted","Data":"0217709df679b58f2acf2bb324edf6f2d54371f94a19fd8a6da21461cc0d318d"}
Oct 30 06:27:25.673893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.673822    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:25.687910 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.687882    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:25.705424 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.705376    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r" podStartSLOduration=1.705363045 podStartE2EDuration="1.705363045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:27:25.649140225 +0000 UTC m=+3349.394425538" watchObservedRunningTime="2026-10-30 06:27:25.705363045 +0000 UTC m=+3349.450648382"
Oct 30 06:27:25.705791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:25.705770    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc" podStartSLOduration=1.705761984 podStartE2EDuration="1.705761984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:27:25.703574575 +0000 UTC m=+3349.448859920" watchObservedRunningTime="2026-10-30 06:27:25.705761984 +0000 UTC m=+3349.451047315"
Oct 30 06:27:26.672013 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:26.671664    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-n57mc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.185:15021/healthz/ready\": dial tcp 10.132.0.185:15021: connect: connection refused" start-of-body=
Oct 30 06:27:26.672013 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:26.671742    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc" podUID="74763aed-8beb-4048-8d84-7e52fa1c19dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.185:15021/healthz/ready\": dial tcp 10.132.0.185:15021: connect: connection refused"
Oct 30 06:27:26.689312 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:26.689279    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-qgv7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.186:15021/healthz/ready\": dial tcp 10.132.0.186:15021: connect: connection refused" start-of-body=
Oct 30 06:27:26.689446 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:26.689329    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r" podUID="4d54096d-00ef-4c39-a0e8-6d1abf5f6640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.186:15021/healthz/ready\": dial tcp 10.132.0.186:15021: connect: connection refused"
Oct 30 06:27:27.669616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:27.668945    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-n57mc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.185:15021/healthz/ready\": dial tcp 10.132.0.185:15021: connect: connection refused" start-of-body=
Oct 30 06:27:27.669616 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:27.669003    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc" podUID="74763aed-8beb-4048-8d84-7e52fa1c19dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.185:15021/healthz/ready\": dial tcp 10.132.0.185:15021: connect: connection refused"
Oct 30 06:27:27.688113 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:27.688068    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-qgv7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.186:15021/healthz/ready\": dial tcp 10.132.0.186:15021: connect: connection refused" start-of-body=
Oct 30 06:27:27.688502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:27.688133    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r" podUID="4d54096d-00ef-4c39-a0e8-6d1abf5f6640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.186:15021/healthz/ready\": dial tcp 10.132.0.186:15021: connect: connection refused"
Oct 30 06:27:28.669613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:28.669466    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-n57mc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.185:15021/healthz/ready\": dial tcp 10.132.0.185:15021: connect: connection refused" start-of-body=
Oct 30 06:27:28.669613 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:28.669527    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc" podUID="74763aed-8beb-4048-8d84-7e52fa1c19dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.185:15021/healthz/ready\": dial tcp 10.132.0.185:15021: connect: connection refused"
Oct 30 06:27:28.688242 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:28.688209    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-qgv7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.186:15021/healthz/ready\": dial tcp 10.132.0.186:15021: connect: connection refused" start-of-body=
Oct 30 06:27:28.688667 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:28.688258    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r" podUID="4d54096d-00ef-4c39-a0e8-6d1abf5f6640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.186:15021/healthz/ready\": dial tcp 10.132.0.186:15021: connect: connection refused"
Oct 30 06:27:29.669614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:29.669010    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-n57mc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.185:15021/healthz/ready\": dial tcp 10.132.0.185:15021: connect: connection refused" start-of-body=
Oct 30 06:27:29.669614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:29.669070    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc" podUID="74763aed-8beb-4048-8d84-7e52fa1c19dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.185:15021/healthz/ready\": dial tcp 10.132.0.185:15021: connect: connection refused"
Oct 30 06:27:29.687903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:29.687869    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-qgv7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.186:15021/healthz/ready\": dial tcp 10.132.0.186:15021: connect: connection refused" start-of-body=
Oct 30 06:27:29.688082 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:29.687918    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r" podUID="4d54096d-00ef-4c39-a0e8-6d1abf5f6640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.186:15021/healthz/ready\": dial tcp 10.132.0.186:15021: connect: connection refused"
Oct 30 06:27:30.671005 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:30.670968    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-n57mc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.185:15021/healthz/ready\": dial tcp 10.132.0.185:15021: connect: connection refused" start-of-body=
Oct 30 06:27:30.671637 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:30.671578    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc" podUID="74763aed-8beb-4048-8d84-7e52fa1c19dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.185:15021/healthz/ready\": dial tcp 10.132.0.185:15021: connect: connection refused"
Oct 30 06:27:30.688328 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:30.688302    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-qgv7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.186:15021/healthz/ready\": dial tcp 10.132.0.186:15021: connect: connection refused" start-of-body=
Oct 30 06:27:30.688533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:30.688513    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r" podUID="4d54096d-00ef-4c39-a0e8-6d1abf5f6640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.186:15021/healthz/ready\": dial tcp 10.132.0.186:15021: connect: connection refused"
Oct 30 06:27:31.668672 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:31.668638    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-n57mc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.185:15021/healthz/ready\": dial tcp 10.132.0.185:15021: connect: connection refused" start-of-body=
Oct 30 06:27:31.668864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:31.668688    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc" podUID="74763aed-8beb-4048-8d84-7e52fa1c19dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.185:15021/healthz/ready\": dial tcp 10.132.0.185:15021: connect: connection refused"
Oct 30 06:27:31.688031 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:31.687997    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-qgv7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.186:15021/healthz/ready\": dial tcp 10.132.0.186:15021: connect: connection refused" start-of-body=
Oct 30 06:27:31.688371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:31.688052    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r" podUID="4d54096d-00ef-4c39-a0e8-6d1abf5f6640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.186:15021/healthz/ready\": dial tcp 10.132.0.186:15021: connect: connection refused"
Oct 30 06:27:32.553516 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:27:32.553468    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:27:33.398710 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:33.398672    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:33.399414 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:33.399393    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:33.400482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:33.400460    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc"
Oct 30 06:27:33.464028 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:33.463998    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:33.464578 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:33.464531    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:33.465708 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:33.465668    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r"
Oct 30 06:27:34.248347 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:34.248309    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:48.247861 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:48.247789    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:27:50.174036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:27:50.174004    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:28:02.551428 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:28:02.551389    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:28:23.880516 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:23.880471    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:28:32.551403 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:28:32.551366    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:28:34.370519 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:34.370454    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:28:35.299112 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:35.299064    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:28:39.565925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.565884    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"]
Oct 30 06:28:39.570296 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.570271    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.616624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.616567    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"]
Oct 30 06:28:39.683973 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.683916    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27hbd\" (UniqueName: \"kubernetes.io/projected/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-kube-api-access-27hbd\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.683973 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.683973    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.684229 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.684089    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.684229 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.684113    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.684229 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.684141    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.684374 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.684262    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.684411 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.684394    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.684457 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.684442    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.684547 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.684531    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.785920 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.785875    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27hbd\" (UniqueName: \"kubernetes.io/projected/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-kube-api-access-27hbd\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.785920 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.785925    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.786187 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.785959    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.786187 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.786094    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.786187 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.786145    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.786187 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.786184    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.786424 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.786261    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.786424 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.786316    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.786547 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.786483    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.786547 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.786522    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.786719 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.786557    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.786782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.786718    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.786849 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.786831    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.786903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.786857    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.788767 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.788748    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.788981 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.788961    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.811721 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.811693    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.811864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.811788    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27hbd\" (UniqueName: \"kubernetes.io/projected/7010a90d-ab6c-4c3c-a5af-f04fea3e6caa-kube-api-access-27hbd\") pod \"maas-default-gateway-openshift-default-58847f9fc5-5cp4f\" (UID: \"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:39.886879 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:39.886797    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:40.205000 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:40.204973    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"]
Oct 30 06:28:40.207334 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:28:40.207305    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7010a90d_ab6c_4c3c_a5af_f04fea3e6caa.slice/crio-f6ca48c80f50daddcdd4a2b5460078740cba335a6d22bade7461842624dbcadf WatchSource:0}: Error finding container f6ca48c80f50daddcdd4a2b5460078740cba335a6d22bade7461842624dbcadf: Status 404 returned error can't find the container with id f6ca48c80f50daddcdd4a2b5460078740cba335a6d22bade7461842624dbcadf
Oct 30 06:28:40.209571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:40.209542    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:28:40.209668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:40.209621    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:28:40.209668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:40.209655    5066 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 06:28:40.964580 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:40.964542    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f" event={"ID":"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa","Type":"ContainerStarted","Data":"f696300f1ee92a7dd4db22d8f9df3bbdb5c0243bdfb6dd037eee36ede855d190"}
Oct 30 06:28:40.965160 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:40.964640    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f" event={"ID":"7010a90d-ab6c-4c3c-a5af-f04fea3e6caa","Type":"ContainerStarted","Data":"f6ca48c80f50daddcdd4a2b5460078740cba335a6d22bade7461842624dbcadf"}
Oct 30 06:28:41.057950 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:41.057901    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f" podStartSLOduration=2.057887203 podStartE2EDuration="2.057887203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:28:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:28:41.04300311 +0000 UTC m=+3424.788288454" watchObservedRunningTime="2026-10-30 06:28:41.057887203 +0000 UTC m=+3424.803172535"
Oct 30 06:28:41.887126 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:41.887088    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:41.888731 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:41.888708    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5cp4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused" start-of-body=
Oct 30 06:28:41.888797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:41.888771    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f" podUID="7010a90d-ab6c-4c3c-a5af-f04fea3e6caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused"
Oct 30 06:28:42.887371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:42.887330    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5cp4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused" start-of-body=
Oct 30 06:28:42.887905 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:42.887392    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f" podUID="7010a90d-ab6c-4c3c-a5af-f04fea3e6caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused"
Oct 30 06:28:43.887438 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:43.887405    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5cp4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused" start-of-body=
Oct 30 06:28:43.887861 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:43.887469    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f" podUID="7010a90d-ab6c-4c3c-a5af-f04fea3e6caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused"
Oct 30 06:28:44.887946 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:44.887902    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5cp4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused" start-of-body=
Oct 30 06:28:44.888619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:44.887957    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f" podUID="7010a90d-ab6c-4c3c-a5af-f04fea3e6caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused"
Oct 30 06:28:45.888091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:45.888053    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5cp4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused" start-of-body=
Oct 30 06:28:45.888775 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:45.888745    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f" podUID="7010a90d-ab6c-4c3c-a5af-f04fea3e6caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused"
Oct 30 06:28:46.887525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:46.887478    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5cp4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused" start-of-body=
Oct 30 06:28:46.887773 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:46.887540    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f" podUID="7010a90d-ab6c-4c3c-a5af-f04fea3e6caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused"
Oct 30 06:28:47.887760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:47.887713    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5cp4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused" start-of-body=
Oct 30 06:28:47.888180 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:47.887781    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f" podUID="7010a90d-ab6c-4c3c-a5af-f04fea3e6caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.187:15021/healthz/ready\": dial tcp 10.132.0.187:15021: connect: connection refused"
Oct 30 06:28:49.927802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:49.927757    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-5cp4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.187:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 06:28:49.928243 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:49.927839    5066 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f" podUID="7010a90d-ab6c-4c3c-a5af-f04fea3e6caa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.187:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 06:28:50.407765 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.407734    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"]
Oct 30 06:28:50.407943 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.407911    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn" podUID="dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" containerName="discovery" containerID="cri-o://203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c" gracePeriod=30
Oct 30 06:28:50.704470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.704446    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 06:28:50.816619 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.816568    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-dns-cert\") pod \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") "
Oct 30 06:28:50.816832 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.816656    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-local-certs\") pod \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") "
Oct 30 06:28:50.816832 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.816689    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-kube-api-access-7j2wt\" (UniqueName: \"kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-kube-api-access-7j2wt\") pod \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") "
Oct 30 06:28:50.816832 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.816770    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-kubeconfig\") pod \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") "
Oct 30 06:28:50.816832 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.816829    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-ca-configmap\") pod \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") "
Oct 30 06:28:50.817076 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.816867    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-cacerts\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-cacerts\") pod \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") "
Oct 30 06:28:50.817076 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.816910    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-token\" (UniqueName: \"kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-token\") pod \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\" (UID: \"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b\") "
Oct 30 06:28:50.817614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.817273    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-ca-configmap" pod "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" (UID: "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:28:50.819284 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.819245    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-dns-cert" pod "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" (UID: "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:28:50.821204 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.821176    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-kubeconfig" pod "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" (UID: "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:28:50.821691 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.821663    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-local-certs" pod "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" (UID: "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:28:50.821691 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.821676    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-cacerts" pod "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" (UID: "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:28:50.822742 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.822716    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-kube-api-access-7j2wt" pod "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" (UID: "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b"). InnerVolumeSpecName "kube-api-access-7j2wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:28:50.831718 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.825722    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-token" pod "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" (UID: "dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:28:50.848458 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.848430    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:50.848771 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.848747    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:50.849960 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.849923    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-5cp4f"
Oct 30 06:28:50.918634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.918582    5066 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-dns-cert\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:28:50.918634 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.918634    5066 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-local-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:28:50.918895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.918651    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j2wt\" (UniqueName: \"kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-kube-api-access-7j2wt\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:28:50.918895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.918665    5066 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-kubeconfig\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:28:50.918895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.918679    5066 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-csr-ca-configmap\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:28:50.918895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.918692    5066 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-cacerts\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:28:50.918895 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:50.918704    5066 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b-istio-token\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:28:51.025802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:51.025711    5066 generic.go:356] "Generic (PLEG): container finished" podID="dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" containerID="203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c" exitCode=0
Oct 30 06:28:51.025802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:51.025781    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"
Oct 30 06:28:51.025802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:51.025788    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn" event={"ID":"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b","Type":"ContainerDied","Data":"203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c"}
Oct 30 06:28:51.026394 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:51.025844    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn" event={"ID":"dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b","Type":"ContainerDied","Data":"01d7bc5313d9b62b37ae54ebe8f5a99dec5d8977229bf754a384ad959ff0541b"}
Oct 30 06:28:51.026394 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:51.025867    5066 scope.go:122] "RemoveContainer" containerID="203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c"
Oct 30 06:28:51.038665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:51.038641    5066 scope.go:122] "RemoveContainer" containerID="203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c"
Oct 30 06:28:51.038916 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:28:51.038898    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c\": container with ID starting with 203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c not found: ID does not exist" containerID="203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c"
Oct 30 06:28:51.038992 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:51.038926    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c"} err="failed to get container status \"203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c\": rpc error: code = NotFound desc = could not find container \"203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c\": container with ID starting with 203810d7ecb1737d0da322cb9fd174aa5fa5078d208a61b697e88aeff8e6a36c not found: ID does not exist"
Oct 30 06:28:51.083866 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:51.083837    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"]
Oct 30 06:28:51.088106 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:51.088084    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d54kn"]
Oct 30 06:28:52.400796 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:28:52.400763    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" path="/var/lib/kubelet/pods/dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b/volumes"
Oct 30 06:29:02.552992 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:29:02.552949    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:29:04.447617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:29:04.446708    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-n57mc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.185:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 06:29:04.447617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:29:04.446779    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc" podUID="74763aed-8beb-4048-8d84-7e52fa1c19dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.185:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 06:29:04.509751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:29:04.509713    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-qgv7r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.186:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 06:29:04.509952 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:29:04.509776    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r" podUID="4d54096d-00ef-4c39-a0e8-6d1abf5f6640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.186:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 06:29:07.887734 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:29:07.887703    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:29:10.855239 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:29:10.855205    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:29:11.013921 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:29:11.013886    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:29:32.552551 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:29:32.552514    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:29:34.498190 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:29:34.498154    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:29:35.426963 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:29:35.426928    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:30:00.142547 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.142508    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"]
Oct 30 06:30:00.143085 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.142986    5066 cpu_manager.go:414] "containerMap: removing container" podUID="dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" containerName="discovery"
Oct 30 06:30:00.143085 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.142998    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" containerName="discovery"
Oct 30 06:30:00.143085 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.143073    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd9d2d88-c2f7-4669-9f6f-6bdf24978b1b" containerName="discovery"
Oct 30 06:30:00.147539 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.147519    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"
Oct 30 06:30:00.165050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.165018    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"]
Oct 30 06:30:00.240793 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.240765    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 06:30:00.240962 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.240764    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 06:30:00.241006 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.240991    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:30:00.322298 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.322264    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpfwd\" (UniqueName: \"kubernetes.io/projected/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-kube-api-access-gpfwd\") pod \"maas-api-key-cleanup-29889030-kvrng\" (UID: \"ab63b18e-5ad9-4686-93cd-51686ee0fd8d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"
Oct 30 06:30:00.322467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.322315    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889030-kvrng\" (UID: \"ab63b18e-5ad9-4686-93cd-51686ee0fd8d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"
Oct 30 06:30:00.422928 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.422897    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpfwd\" (UniqueName: \"kubernetes.io/projected/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-kube-api-access-gpfwd\") pod \"maas-api-key-cleanup-29889030-kvrng\" (UID: \"ab63b18e-5ad9-4686-93cd-51686ee0fd8d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"
Oct 30 06:30:00.423129 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.422964    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889030-kvrng\" (UID: \"ab63b18e-5ad9-4686-93cd-51686ee0fd8d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"
Oct 30 06:30:00.423668 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.423643    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889030-kvrng\" (UID: \"ab63b18e-5ad9-4686-93cd-51686ee0fd8d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"
Oct 30 06:30:00.449325 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.449291    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpfwd\" (UniqueName: \"kubernetes.io/projected/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-kube-api-access-gpfwd\") pod \"maas-api-key-cleanup-29889030-kvrng\" (UID: \"ab63b18e-5ad9-4686-93cd-51686ee0fd8d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"
Oct 30 06:30:00.462690 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.462665    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"
Oct 30 06:30:00.761365 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:00.761339    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"]
Oct 30 06:30:00.763430 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:30:00.763399    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab63b18e_5ad9_4686_93cd_51686ee0fd8d.slice/crio-e6bb13b3fcee492c7f0fe7c82335d37a33319cfe365e136329b532d597e6240b WatchSource:0}: Error finding container e6bb13b3fcee492c7f0fe7c82335d37a33319cfe365e136329b532d597e6240b: Status 404 returned error can't find the container with id e6bb13b3fcee492c7f0fe7c82335d37a33319cfe365e136329b532d597e6240b
Oct 30 06:30:01.441713 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:01.441680    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng" event={"ID":"ab63b18e-5ad9-4686-93cd-51686ee0fd8d","Type":"ContainerStarted","Data":"ba791b4e817713ba06c95ab40fbc9c0f766ee0042a1efae6ecd5f50b603ad9a8"}
Oct 30 06:30:01.442110 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:01.441731    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng" event={"ID":"ab63b18e-5ad9-4686-93cd-51686ee0fd8d","Type":"ContainerStarted","Data":"e6bb13b3fcee492c7f0fe7c82335d37a33319cfe365e136329b532d597e6240b"}
Oct 30 06:30:01.486874 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:01.486826    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng" podStartSLOduration=1.48681276 podStartE2EDuration="1.48681276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:30:01.485243969 +0000 UTC m=+3505.230529316" watchObservedRunningTime="2026-10-30 06:30:01.48681276 +0000 UTC m=+3505.232098092"
Oct 30 06:30:02.455524 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:02.455486    5066 generic.go:356] "Generic (PLEG): container finished" podID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" containerID="ba791b4e817713ba06c95ab40fbc9c0f766ee0042a1efae6ecd5f50b603ad9a8" exitCode=60
Oct 30 06:30:02.455963 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:02.455578    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng" event={"ID":"ab63b18e-5ad9-4686-93cd-51686ee0fd8d","Type":"ContainerDied","Data":"ba791b4e817713ba06c95ab40fbc9c0f766ee0042a1efae6ecd5f50b603ad9a8"}
Oct 30 06:30:02.455963 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:02.455774    5066 scope.go:122] "RemoveContainer" containerID="ba791b4e817713ba06c95ab40fbc9c0f766ee0042a1efae6ecd5f50b603ad9a8"
Oct 30 06:30:02.552520 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:30:02.552475    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:30:03.462996 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:03.462968    5066 generic.go:356] "Generic (PLEG): container finished" podID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" containerID="5e104133d117efb4429b873cf1eb08a01d47c70104c5eaefa5802f71c32e75eb" exitCode=60
Oct 30 06:30:03.463461 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:03.463021    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng" event={"ID":"ab63b18e-5ad9-4686-93cd-51686ee0fd8d","Type":"ContainerDied","Data":"5e104133d117efb4429b873cf1eb08a01d47c70104c5eaefa5802f71c32e75eb"}
Oct 30 06:30:03.463461 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:03.463065    5066 scope.go:122] "RemoveContainer" containerID="ba791b4e817713ba06c95ab40fbc9c0f766ee0042a1efae6ecd5f50b603ad9a8"
Oct 30 06:30:03.463461 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:03.463283    5066 scope.go:122] "RemoveContainer" containerID="5e104133d117efb4429b873cf1eb08a01d47c70104c5eaefa5802f71c32e75eb"
Oct 30 06:30:03.463663 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:30:03.463528    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889030-kvrng_odh-ai-gateway-infra(ab63b18e-5ad9-4686-93cd-51686ee0fd8d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d"
Oct 30 06:30:04.469526 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:04.469491    5066 scope.go:122] "RemoveContainer" containerID="5e104133d117efb4429b873cf1eb08a01d47c70104c5eaefa5802f71c32e75eb"
Oct 30 06:30:04.470067 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:30:04.469784    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889030-kvrng_odh-ai-gateway-infra(ab63b18e-5ad9-4686-93cd-51686ee0fd8d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d"
Oct 30 06:30:17.394807 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:17.394776    5066 scope.go:122] "RemoveContainer" containerID="5e104133d117efb4429b873cf1eb08a01d47c70104c5eaefa5802f71c32e75eb"
Oct 30 06:30:18.553410 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:18.553331    5066 generic.go:356] "Generic (PLEG): container finished" podID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" containerID="c420b45de96748cea20d5dc3fbe63910f02172d0d5c580555e737958b3c51258" exitCode=60
Oct 30 06:30:18.553410 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:18.553388    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng" event={"ID":"ab63b18e-5ad9-4686-93cd-51686ee0fd8d","Type":"ContainerDied","Data":"c420b45de96748cea20d5dc3fbe63910f02172d0d5c580555e737958b3c51258"}
Oct 30 06:30:18.553917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:18.553437    5066 scope.go:122] "RemoveContainer" containerID="5e104133d117efb4429b873cf1eb08a01d47c70104c5eaefa5802f71c32e75eb"
Oct 30 06:30:18.553917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:18.553837    5066 scope.go:122] "RemoveContainer" containerID="c420b45de96748cea20d5dc3fbe63910f02172d0d5c580555e737958b3c51258"
Oct 30 06:30:18.554118 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:30:18.554094    5066 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889030-kvrng_odh-ai-gateway-infra(ab63b18e-5ad9-4686-93cd-51686ee0fd8d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d"
Oct 30 06:30:19.607289 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:19.607256    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"]
Oct 30 06:30:19.752423 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:19.752400    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"
Oct 30 06:30:19.842091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:19.842056    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-kube-api-access-gpfwd\" (UniqueName: \"kubernetes.io/projected/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-kube-api-access-gpfwd\") pod \"ab63b18e-5ad9-4686-93cd-51686ee0fd8d\" (UID: \"ab63b18e-5ad9-4686-93cd-51686ee0fd8d\") "
Oct 30 06:30:19.842302 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:19.842149    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-maas-api-service-ca\") pod \"ab63b18e-5ad9-4686-93cd-51686ee0fd8d\" (UID: \"ab63b18e-5ad9-4686-93cd-51686ee0fd8d\") "
Oct 30 06:30:19.842539 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:19.842511    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-maas-api-service-ca" pod "ab63b18e-5ad9-4686-93cd-51686ee0fd8d" (UID: "ab63b18e-5ad9-4686-93cd-51686ee0fd8d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:30:19.852095 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:19.852045    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-kube-api-access-gpfwd" pod "ab63b18e-5ad9-4686-93cd-51686ee0fd8d" (UID: "ab63b18e-5ad9-4686-93cd-51686ee0fd8d"). InnerVolumeSpecName "kube-api-access-gpfwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:30:19.943352 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:19.943319    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpfwd\" (UniqueName: \"kubernetes.io/projected/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-kube-api-access-gpfwd\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:30:19.943352 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:19.943350    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab63b18e-5ad9-4686-93cd-51686ee0fd8d-maas-api-service-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:30:20.566456 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:20.566421    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng" event={"ID":"ab63b18e-5ad9-4686-93cd-51686ee0fd8d","Type":"ContainerDied","Data":"e6bb13b3fcee492c7f0fe7c82335d37a33319cfe365e136329b532d597e6240b"}
Oct 30 06:30:20.566675 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:20.566478    5066 scope.go:122] "RemoveContainer" containerID="c420b45de96748cea20d5dc3fbe63910f02172d0d5c580555e737958b3c51258"
Oct 30 06:30:20.566675 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:20.566433    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"
Oct 30 06:30:20.625496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:20.625470    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"]
Oct 30 06:30:20.639325 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:20.639294    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889030-kvrng"]
Oct 30 06:30:22.400643 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:22.400583    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" path="/var/lib/kubelet/pods/ab63b18e-5ad9-4686-93cd-51686ee0fd8d/volumes"
Oct 30 06:30:32.551502 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:30:32.551472    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:30:34.656119 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:34.656089    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:30:35.569523 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:30:35.569487    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:31:02.551172 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:31:02.551139    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:31:32.550965 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:31:32.550931    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:31:34.762816 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:31:34.762781    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:31:35.685066 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:31:35.685021    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:31:36.981492 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:31:36.981461    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:31:37.000518 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:31:37.000494    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:31:38.191962 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:31:38.191929    5066 scope.go:122] "RemoveContainer" containerID="72672ead92272af0557d83dab390d5f95ef743c453bde70b768dbe21a676a5e3"
Oct 30 06:31:49.028892 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:31:49.028852    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"]
Oct 30 06:31:49.044046 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:31:49.044010    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c3562a41-cq24s"]
Oct 30 06:31:50.400276 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:31:50.400241    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b80e39d8-daad-4c50-aabb-458e5505dcde" path="/var/lib/kubelet/pods/b80e39d8-daad-4c50-aabb-458e5505dcde/volumes"
Oct 30 06:32:02.551812 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:32:02.551765    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:32:22.889353 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:22.889262    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:24.657815 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:24.657773    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:26.759848 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:26.759811    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:26.928888 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:26.928856    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:28.185148 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:28.185108    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:28.355459 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:28.355419    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:28.526948 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:28.526871    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:29.856770 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:29.856734    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:31.467438 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:31.467407    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:32.551720 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:32:32.551686    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:32:33.059898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:33.059868    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:34.678577 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:34.678548    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:34.879733 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:34.879703    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:35.813199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:35.813163    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:36.151158 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:36.151075    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:37.691112 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:37.691081    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:39.216644 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:39.216606    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:40.675046 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:40.675011    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:41.343154 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:41.343124    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:41.693550 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:41.693515    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:42.320384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:42.320351    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:42.502209 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:42.502181    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:42.664055 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:42.664030    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:44.041155 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.041113    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:44.445264 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.445229    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"]
Oct 30 06:32:44.445660 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.445646    5066 cpu_manager.go:414] "containerMap: removing container" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" containerName="cleanup"
Oct 30 06:32:44.445711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.445661    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" containerName="cleanup"
Oct 30 06:32:44.445711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.445691    5066 cpu_manager.go:414] "containerMap: removing container" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" containerName="cleanup"
Oct 30 06:32:44.445711 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.445696    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" containerName="cleanup"
Oct 30 06:32:44.445813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.445763    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" containerName="cleanup"
Oct 30 06:32:44.445813 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.445774    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" containerName="cleanup"
Oct 30 06:32:44.449935 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.449913    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.469161 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.469112    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"]
Oct 30 06:32:44.481469 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.481418    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-6c0c09\"/\"kube-root-ca.crt\""
Oct 30 06:32:44.481469 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.481418    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-6c0c09\"/\"openshift-service-ca.crt\""
Oct 30 06:32:44.527446 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.527409    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-6c0c09\"/\"default-dockercfg-jc6tq\""
Oct 30 06:32:44.527446 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.527409    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-6c0c09\"/\"simulator-single-main-6c0c09-kserve-self-signed-certs\""
Oct 30 06:32:44.555721 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.555689    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-home\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.555893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.555809    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mdlt\" (UniqueName: \"kubernetes.io/projected/91a2c6a2-d223-410a-b992-9e568953021e-kube-api-access-2mdlt\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.555893 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.555856    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-model-cache\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.556008 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.555900    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-dshm\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.556069 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.556050    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a2c6a2-d223-410a-b992-9e568953021e-tls-certs\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.556114 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.556094    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-tmp-dir\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.576965 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.576931    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:44.657476 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.657442    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-home\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.657698 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.657487    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mdlt\" (UniqueName: \"kubernetes.io/projected/91a2c6a2-d223-410a-b992-9e568953021e-kube-api-access-2mdlt\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.657698 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.657638    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-model-cache\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.657698 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.657683    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-dshm\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.657837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.657748    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a2c6a2-d223-410a-b992-9e568953021e-tls-certs\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.657885 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.657849    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-tmp-dir\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.657885 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.657866    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-home\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.658050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.658027    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-model-cache\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.658151 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.658128    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-tmp-dir\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.660187 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.660160    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-dshm\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.660354 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.660335    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a2c6a2-d223-410a-b992-9e568953021e-tls-certs\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.683384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.683357    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mdlt\" (UniqueName: \"kubernetes.io/projected/91a2c6a2-d223-410a-b992-9e568953021e-kube-api-access-2mdlt\") pod \"simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") " pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:44.777754 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:44.777660    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:45.081689 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:45.081648    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"]
Oct 30 06:32:45.351003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:45.350900    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc" event={"ID":"91a2c6a2-d223-410a-b992-9e568953021e","Type":"ContainerStarted","Data":"b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29"}
Oct 30 06:32:45.351003 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:45.350952    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc" event={"ID":"91a2c6a2-d223-410a-b992-9e568953021e","Type":"ContainerStarted","Data":"5bcd76ba7f7815ad1a7fb8004f495c13435da612fe38e8199710951f5fda0b84"}
Oct 30 06:32:45.400552 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:45.400505    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc" podStartSLOduration=1.4004902559999999 podStartE2EDuration="1.400490256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:32:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:32:45.399212611 +0000 UTC m=+3669.144497944" watchObservedRunningTime="2026-10-30 06:32:45.400490256 +0000 UTC m=+3669.145775606"
Oct 30 06:32:45.767623 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:45.767576    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:45.778528 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:45.778495    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:47.505215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:47.505180    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:49.075666 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:49.075636    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:50.512865 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:50.512832    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:52.052870 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:52.052834    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:53.592386 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:53.592351    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:54.778667 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:54.778629    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:54.791531 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:54.791495    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:55.110433 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:55.110342    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:55.413172 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:55.413143    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:32:56.667869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:56.667824    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:58.176091 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:58.176054    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:32:59.791750 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:32:59.791696    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:01.566806 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:01.566776    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:02.552247 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:33:02.552212    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:33:03.067611 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:03.067560    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:04.498519 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:04.498482    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:06.086972 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:06.086926    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:07.419947 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.419916    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"]
Oct 30 06:33:07.420374 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.420359    5066 cpu_manager.go:414] "containerMap: removing container" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" containerName="cleanup"
Oct 30 06:33:07.420419 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.420375    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" containerName="cleanup"
Oct 30 06:33:07.420480 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.420470    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab63b18e-5ad9-4686-93cd-51686ee0fd8d" containerName="cleanup"
Oct 30 06:33:07.423788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.423768    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:07.441826 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.441803    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 06:33:07.455684 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.455656    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 06:33:07.455868 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.455683    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 06:33:07.455868 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.455691    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:33:07.458047 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.458025    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"]
Oct 30 06:33:07.483020 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.482992    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fhm8\" (UniqueName: \"kubernetes.io/projected/b5ea76c7-2c83-4499-8b15-21d7c16d3727-kube-api-access-2fhm8\") pod \"maas-api-revoke-sub-6a85919a-zkgkq\" (UID: \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:07.483173 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.483043    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6a85919a-zkgkq\" (UID: \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:07.483223 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.483186    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6a85919a-zkgkq\" (UID: \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:07.529487 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.529449    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:07.583750 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.583711    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6a85919a-zkgkq\" (UID: \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:07.583969 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.583792    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fhm8\" (UniqueName: \"kubernetes.io/projected/b5ea76c7-2c83-4499-8b15-21d7c16d3727-kube-api-access-2fhm8\") pod \"maas-api-revoke-sub-6a85919a-zkgkq\" (UID: \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:07.583969 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.583835    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6a85919a-zkgkq\" (UID: \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:07.584534 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.584460    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6a85919a-zkgkq\" (UID: \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:07.586379 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.586351    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6a85919a-zkgkq\" (UID: \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:07.609259 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.609227    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fhm8\" (UniqueName: \"kubernetes.io/projected/b5ea76c7-2c83-4499-8b15-21d7c16d3727-kube-api-access-2fhm8\") pod \"maas-api-revoke-sub-6a85919a-zkgkq\" (UID: \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:07.737548 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:07.737465    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:08.037659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:08.037635    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"]
Oct 30 06:33:08.039056 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:33:08.039024    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5ea76c7_2c83_4499_8b15_21d7c16d3727.slice/crio-7051adf1e0834d1cba86460bf2df68bb52aa61cf138fb929057e0a7041252fd5 WatchSource:0}: Error finding container 7051adf1e0834d1cba86460bf2df68bb52aa61cf138fb929057e0a7041252fd5: Status 404 returned error can't find the container with id 7051adf1e0834d1cba86460bf2df68bb52aa61cf138fb929057e0a7041252fd5
Oct 30 06:33:08.467234 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:08.467200    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq" event={"ID":"b5ea76c7-2c83-4499-8b15-21d7c16d3727","Type":"ContainerStarted","Data":"ab350ab9ab0b904fa8ad9e09744e107636f63c9c4fff703814849ddabf23fa8d"}
Oct 30 06:33:08.467680 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:08.467262    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq" event={"ID":"b5ea76c7-2c83-4499-8b15-21d7c16d3727","Type":"ContainerStarted","Data":"7051adf1e0834d1cba86460bf2df68bb52aa61cf138fb929057e0a7041252fd5"}
Oct 30 06:33:08.512008 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:08.511961    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq" podStartSLOduration=1.511947449 podStartE2EDuration="1.511947449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:33:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:33:08.510842075 +0000 UTC m=+3692.256127408" watchObservedRunningTime="2026-10-30 06:33:08.511947449 +0000 UTC m=+3692.257232781"
Oct 30 06:33:09.051830 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:09.051757    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:09.473334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:09.473293    5066 generic.go:356] "Generic (PLEG): container finished" podID="b5ea76c7-2c83-4499-8b15-21d7c16d3727" containerID="ab350ab9ab0b904fa8ad9e09744e107636f63c9c4fff703814849ddabf23fa8d" exitCode=0
Oct 30 06:33:09.473790 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:09.473377    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq" event={"ID":"b5ea76c7-2c83-4499-8b15-21d7c16d3727","Type":"ContainerDied","Data":"ab350ab9ab0b904fa8ad9e09744e107636f63c9c4fff703814849ddabf23fa8d"}
Oct 30 06:33:10.573788 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:10.573755    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:10.623290 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:10.623260    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:10.717228 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:10.717195    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-openshift-ca\") pod \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\" (UID: \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\") "
Oct 30 06:33:10.717400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:10.717238    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-xks-ca\") pod \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\" (UID: \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\") "
Oct 30 06:33:10.717400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:10.717361    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5ea76c7-2c83-4499-8b15-21d7c16d3727-kube-api-access-2fhm8\" (UniqueName: \"kubernetes.io/projected/b5ea76c7-2c83-4499-8b15-21d7c16d3727-kube-api-access-2fhm8\") pod \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\" (UID: \"b5ea76c7-2c83-4499-8b15-21d7c16d3727\") "
Oct 30 06:33:10.717659 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:10.717635    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-openshift-ca" pod "b5ea76c7-2c83-4499-8b15-21d7c16d3727" (UID: "b5ea76c7-2c83-4499-8b15-21d7c16d3727"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:33:10.719605 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:10.719573    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5ea76c7-2c83-4499-8b15-21d7c16d3727-kube-api-access-2fhm8" pod "b5ea76c7-2c83-4499-8b15-21d7c16d3727" (UID: "b5ea76c7-2c83-4499-8b15-21d7c16d3727"). InnerVolumeSpecName "kube-api-access-2fhm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:33:10.719706 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:10.719579    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-xks-ca" pod "b5ea76c7-2c83-4499-8b15-21d7c16d3727" (UID: "b5ea76c7-2c83-4499-8b15-21d7c16d3727"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:33:10.818217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:10.818119    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:33:10.818217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:10.818149    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b5ea76c7-2c83-4499-8b15-21d7c16d3727-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:33:10.818217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:10.818158    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fhm8\" (UniqueName: \"kubernetes.io/projected/b5ea76c7-2c83-4499-8b15-21d7c16d3727-kube-api-access-2fhm8\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:33:11.485210 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:11.485182    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"
Oct 30 06:33:11.485210 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:11.485189    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq" event={"ID":"b5ea76c7-2c83-4499-8b15-21d7c16d3727","Type":"ContainerDied","Data":"7051adf1e0834d1cba86460bf2df68bb52aa61cf138fb929057e0a7041252fd5"}
Oct 30 06:33:11.485456 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:11.485255    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7051adf1e0834d1cba86460bf2df68bb52aa61cf138fb929057e0a7041252fd5"
Oct 30 06:33:12.021658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:12.021550    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:13.575612 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:13.575556    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:15.035801 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:15.035768    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:16.551138 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:16.551101    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:18.169942 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.169902    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:18.545303 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.545226    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:18.658272 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.658231    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"]
Oct 30 06:33:18.659237 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.659208    5066 cpu_manager.go:414] "containerMap: removing container" podUID="b5ea76c7-2c83-4499-8b15-21d7c16d3727" containerName="revoke-keys"
Oct 30 06:33:18.659237 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.659237    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5ea76c7-2c83-4499-8b15-21d7c16d3727" containerName="revoke-keys"
Oct 30 06:33:18.659496 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.659481    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5ea76c7-2c83-4499-8b15-21d7c16d3727" containerName="revoke-keys"
Oct 30 06:33:18.664195 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.664168    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:18.682625 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.682579    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"]
Oct 30 06:33:18.694176 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.694153    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 06:33:18.694345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.694223    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 06:33:18.694345 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.694234    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:33:18.695463 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.695445    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 06:33:18.794044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.794010    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl\" (UID: \"89f9384f-736e-422b-9f19-457d5930c91f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:18.794239 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.794164    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl\" (UID: \"89f9384f-736e-422b-9f19-457d5930c91f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:18.794239 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.794197    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tfx5\" (UniqueName: \"kubernetes.io/projected/89f9384f-736e-422b-9f19-457d5930c91f-kube-api-access-5tfx5\") pod \"maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl\" (UID: \"89f9384f-736e-422b-9f19-457d5930c91f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:18.895841 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.895754    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl\" (UID: \"89f9384f-736e-422b-9f19-457d5930c91f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:18.895841 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.895838    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl\" (UID: \"89f9384f-736e-422b-9f19-457d5930c91f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:18.896031 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.895857    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tfx5\" (UniqueName: \"kubernetes.io/projected/89f9384f-736e-422b-9f19-457d5930c91f-kube-api-access-5tfx5\") pod \"maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl\" (UID: \"89f9384f-736e-422b-9f19-457d5930c91f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:18.896637 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.896540    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl\" (UID: \"89f9384f-736e-422b-9f19-457d5930c91f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:18.898412 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.898390    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl\" (UID: \"89f9384f-736e-422b-9f19-457d5930c91f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:18.921504 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.921477    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tfx5\" (UniqueName: \"kubernetes.io/projected/89f9384f-736e-422b-9f19-457d5930c91f-kube-api-access-5tfx5\") pod \"maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl\" (UID: \"89f9384f-736e-422b-9f19-457d5930c91f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:18.977663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:18.977634    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:19.006966 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:19.006933    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:19.162934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:19.162907    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:19.278038 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:19.278014    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"]
Oct 30 06:33:19.280138 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:33:19.280106    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89f9384f_736e_422b_9f19_457d5930c91f.slice/crio-8f39cddbbcb361429c5a922134fc774c4125472727c7a880832953f0938f9913 WatchSource:0}: Error finding container 8f39cddbbcb361429c5a922134fc774c4125472727c7a880832953f0938f9913: Status 404 returned error can't find the container with id 8f39cddbbcb361429c5a922134fc774c4125472727c7a880832953f0938f9913
Oct 30 06:33:19.531836 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:19.531808    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl" event={"ID":"89f9384f-736e-422b-9f19-457d5930c91f","Type":"ContainerStarted","Data":"8f39cddbbcb361429c5a922134fc774c4125472727c7a880832953f0938f9913"}
Oct 30 06:33:19.856326 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:19.856240    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:20.537824 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:20.537790    5066 generic.go:356] "Generic (PLEG): container finished" podID="89f9384f-736e-422b-9f19-457d5930c91f" containerID="3751c1650429c026a4c80069aea882963d1fc46c62104f7e373af3a08b11c5b3" exitCode=0
Oct 30 06:33:20.538248 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:20.537882    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl" event={"ID":"89f9384f-736e-422b-9f19-457d5930c91f","Type":"ContainerDied","Data":"3751c1650429c026a4c80069aea882963d1fc46c62104f7e373af3a08b11c5b3"}
Oct 30 06:33:21.346278 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:21.346242    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:21.680046 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:21.680014    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:21.830751 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:21.830717    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89f9384f-736e-422b-9f19-457d5930c91f-kube-api-access-5tfx5\" (UniqueName: \"kubernetes.io/projected/89f9384f-736e-422b-9f19-457d5930c91f-kube-api-access-5tfx5\") pod \"89f9384f-736e-422b-9f19-457d5930c91f\" (UID: \"89f9384f-736e-422b-9f19-457d5930c91f\") "
Oct 30 06:33:21.830938 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:21.830768    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-openshift-ca\") pod \"89f9384f-736e-422b-9f19-457d5930c91f\" (UID: \"89f9384f-736e-422b-9f19-457d5930c91f\") "
Oct 30 06:33:21.830938 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:21.830822    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-xks-ca\") pod \"89f9384f-736e-422b-9f19-457d5930c91f\" (UID: \"89f9384f-736e-422b-9f19-457d5930c91f\") "
Oct 30 06:33:21.831158 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:21.831128    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-openshift-ca" pod "89f9384f-736e-422b-9f19-457d5930c91f" (UID: "89f9384f-736e-422b-9f19-457d5930c91f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:33:21.833060 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:21.833038    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89f9384f-736e-422b-9f19-457d5930c91f-kube-api-access-5tfx5" pod "89f9384f-736e-422b-9f19-457d5930c91f" (UID: "89f9384f-736e-422b-9f19-457d5930c91f"). InnerVolumeSpecName "kube-api-access-5tfx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:33:21.833154 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:21.833093    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-xks-ca" pod "89f9384f-736e-422b-9f19-457d5930c91f" (UID: "89f9384f-736e-422b-9f19-457d5930c91f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:33:21.932797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:21.932714    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tfx5\" (UniqueName: \"kubernetes.io/projected/89f9384f-736e-422b-9f19-457d5930c91f-kube-api-access-5tfx5\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:33:21.932797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:21.932741    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:33:21.932797 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:21.932752    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/89f9384f-736e-422b-9f19-457d5930c91f-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:33:22.548842 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:22.548803    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl" event={"ID":"89f9384f-736e-422b-9f19-457d5930c91f","Type":"ContainerDied","Data":"8f39cddbbcb361429c5a922134fc774c4125472727c7a880832953f0938f9913"}
Oct 30 06:33:22.548842 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:22.548840    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"
Oct 30 06:33:22.549050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:22.548863    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f39cddbbcb361429c5a922134fc774c4125472727c7a880832953f0938f9913"
Oct 30 06:33:22.832908 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:22.832823    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:23.545825 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:23.545787    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"]
Oct 30 06:33:23.683440 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:23.683408    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:24.377731 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:24.377672    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:25.838258 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:25.838215    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:27.368071 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:27.368017    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:28.846334 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:28.846303    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:29.245086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:29.245027    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:29.402199 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:29.402163    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:30.225883 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:30.225847    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:30.769967 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:30.769931    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:30.811812 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:30.811779    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"]
Oct 30 06:33:30.825799 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:30.825770    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-6c0c09-2qqzl"]
Oct 30 06:33:30.938544 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:30.938518    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:31.373120 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:31.373088    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:32.400830 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:32.400799    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89f9384f-736e-422b-9f19-457d5930c91f" path="/var/lib/kubelet/pods/89f9384f-736e-422b-9f19-457d5930c91f/volumes"
Oct 30 06:33:32.559817 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:33:32.559783    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:33:34.999709 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:34.999667    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:35.929753 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:35.929718    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:37.860034 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:37.860005    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:38.016331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.016300    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:38.212671 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.212635    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:33:38.223791 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.223760    5066 scope.go:122] "RemoveContainer" containerID="b1d1715fccd9251b1912cd5c1e97b915ddd4b69544c9f56b2814f4452b29459e"
Oct 30 06:33:38.566761 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.566664    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc" podUID="91a2c6a2-d223-410a-b992-9e568953021e" containerName="main" containerID="cri-o://b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29" gracePeriod=45
Oct 30 06:33:38.828354 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.828282    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:33:38.893781 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.893733    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-home\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-home\") pod \"91a2c6a2-d223-410a-b992-9e568953021e\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") "
Oct 30 06:33:38.894896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.893881    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-tmp-dir\") pod \"91a2c6a2-d223-410a-b992-9e568953021e\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") "
Oct 30 06:33:38.894896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.893938    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-dshm\") pod \"91a2c6a2-d223-410a-b992-9e568953021e\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") "
Oct 30 06:33:38.894896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.893978    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91a2c6a2-d223-410a-b992-9e568953021e-tls-certs\" (UniqueName: \"kubernetes.io/secret/91a2c6a2-d223-410a-b992-9e568953021e-tls-certs\") pod \"91a2c6a2-d223-410a-b992-9e568953021e\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") "
Oct 30 06:33:38.894896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.894021    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91a2c6a2-d223-410a-b992-9e568953021e-kube-api-access-2mdlt\" (UniqueName: \"kubernetes.io/projected/91a2c6a2-d223-410a-b992-9e568953021e-kube-api-access-2mdlt\") pod \"91a2c6a2-d223-410a-b992-9e568953021e\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") "
Oct 30 06:33:38.894896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.894087    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-model-cache\") pod \"91a2c6a2-d223-410a-b992-9e568953021e\" (UID: \"91a2c6a2-d223-410a-b992-9e568953021e\") "
Oct 30 06:33:38.894896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.894635    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-model-cache" pod "91a2c6a2-d223-410a-b992-9e568953021e" (UID: "91a2c6a2-d223-410a-b992-9e568953021e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:33:38.894896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.894832    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-home" pod "91a2c6a2-d223-410a-b992-9e568953021e" (UID: "91a2c6a2-d223-410a-b992-9e568953021e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:33:38.895171 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.895002    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-tmp-dir" pod "91a2c6a2-d223-410a-b992-9e568953021e" (UID: "91a2c6a2-d223-410a-b992-9e568953021e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:33:38.897471 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.897439    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-dshm" pod "91a2c6a2-d223-410a-b992-9e568953021e" (UID: "91a2c6a2-d223-410a-b992-9e568953021e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:33:38.897682 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.897490    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91a2c6a2-d223-410a-b992-9e568953021e-kube-api-access-2mdlt" pod "91a2c6a2-d223-410a-b992-9e568953021e" (UID: "91a2c6a2-d223-410a-b992-9e568953021e"). InnerVolumeSpecName "kube-api-access-2mdlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:33:38.897970 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.897945    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91a2c6a2-d223-410a-b992-9e568953021e-tls-certs" pod "91a2c6a2-d223-410a-b992-9e568953021e" (UID: "91a2c6a2-d223-410a-b992-9e568953021e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:33:38.995068 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.995036    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:33:38.995068 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.995064    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a2c6a2-d223-410a-b992-9e568953021e-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:33:38.995068 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.995075    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mdlt\" (UniqueName: \"kubernetes.io/projected/91a2c6a2-d223-410a-b992-9e568953021e-kube-api-access-2mdlt\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:33:38.995309 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.995085    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:33:38.995309 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.995095    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:33:38.995309 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:38.995123    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a2c6a2-d223-410a-b992-9e568953021e-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:33:39.635910 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:39.635881    5066 generic.go:356] "Generic (PLEG): container finished" podID="91a2c6a2-d223-410a-b992-9e568953021e" containerID="b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29" exitCode=0
Oct 30 06:33:39.636137 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:39.635926    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"
Oct 30 06:33:39.636137 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:39.635964    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc" event={"ID":"91a2c6a2-d223-410a-b992-9e568953021e","Type":"ContainerDied","Data":"b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29"}
Oct 30 06:33:39.636137 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:39.636012    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc" event={"ID":"91a2c6a2-d223-410a-b992-9e568953021e","Type":"ContainerDied","Data":"5bcd76ba7f7815ad1a7fb8004f495c13435da612fe38e8199710951f5fda0b84"}
Oct 30 06:33:39.636137 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:39.636036    5066 scope.go:122] "RemoveContainer" containerID="b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29"
Oct 30 06:33:39.646449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:39.646432    5066 scope.go:122] "RemoveContainer" containerID="b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29"
Oct 30 06:33:39.646738 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:33:39.646722    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29\": container with ID starting with b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29 not found: ID does not exist" containerID="b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29"
Oct 30 06:33:39.646793 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:39.646746    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29"} err="failed to get container status \"b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29\": rpc error: code = NotFound desc = could not find container \"b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29\": container with ID starting with b7c87ea1f6463d2e237c45b42a447e59c4e55465a77b277b27811cca96792c29 not found: ID does not exist"
Oct 30 06:33:39.692406 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:39.692373    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"]
Oct 30 06:33:39.706645 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:39.706614    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-6c0c09/simulator-single-main-6c0c09-kserve-dfd5547c5-xzdvc"]
Oct 30 06:33:40.400336 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:33:40.400307    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91a2c6a2-d223-410a-b992-9e568953021e" path="/var/lib/kubelet/pods/91a2c6a2-d223-410a-b992-9e568953021e/volumes"
Oct 30 06:34:00.189883 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:34:00.189791    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:34:01.218826 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:34:01.218791    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:34:02.551167 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:34:02.551131    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:34:07.124846 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:34:07.124809    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:34:32.552176 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:34:32.552132    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:34:35.128249 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:34:35.128210    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:34:36.062480 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:34:36.062445    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:35:02.551006 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:35:02.550974    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:35:19.934766 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:35:19.934684    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:35:32.554446 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:35:32.554412    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:35:35.300116 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:35:35.300082    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:35:36.183766 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:35:36.183731    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:35:56.012801 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:35:56.012751    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:35:56.502639 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:35:56.494774    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:35:56.661843 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:35:56.661809    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:35:59.672107 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:35:59.672077    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:35:59.841134 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:35:59.841092    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:01.053197 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:01.053161    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:01.224557 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:01.224525    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:01.388292 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:01.388216    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:02.552024 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:36:02.551986    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:36:02.878135 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:02.878045    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:04.375852 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:04.375819    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:05.969582 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:05.969554    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:07.554966 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:07.554923    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:08.931274 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:08.931237    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:09.094555 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:09.094526    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:10.667967 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:10.667930    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:12.213679 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:12.213641    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:13.822919 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:13.822886    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:14.599154 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:14.599123    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:15.205903 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:15.205872    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:15.382579 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:15.382544    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:16.716527 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.716489    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"]
Oct 30 06:36:16.717057 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.716971    5066 cpu_manager.go:414] "containerMap: removing container" podUID="89f9384f-736e-422b-9f19-457d5930c91f" containerName="revoke-keys"
Oct 30 06:36:16.717057 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.716983    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89f9384f-736e-422b-9f19-457d5930c91f" containerName="revoke-keys"
Oct 30 06:36:16.717057 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.717005    5066 cpu_manager.go:414] "containerMap: removing container" podUID="91a2c6a2-d223-410a-b992-9e568953021e" containerName="main"
Oct 30 06:36:16.717057 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.717011    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91a2c6a2-d223-410a-b992-9e568953021e" containerName="main"
Oct 30 06:36:16.717294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.717099    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="89f9384f-736e-422b-9f19-457d5930c91f" containerName="revoke-keys"
Oct 30 06:36:16.717294 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.717109    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="91a2c6a2-d223-410a-b992-9e568953021e" containerName="main"
Oct 30 06:36:16.720427 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.720395    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.756826 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.756781    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-ee1c63\"/\"kube-root-ca.crt\""
Oct 30 06:36:16.757042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.756843    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-ee1c63\"/\"default-dockercfg-47vq6\""
Oct 30 06:36:16.757042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.756843    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-ee1c63\"/\"simulator-single-main-ee1c63-kserve-self-signed-certs\""
Oct 30 06:36:16.757042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.756888    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-ee1c63\"/\"openshift-service-ca.crt\""
Oct 30 06:36:16.759746 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.759725    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"]
Oct 30 06:36:16.819802 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.819772    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-home\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.820002 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.819833    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-dshm\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.820002 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.819862    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-tmp-dir\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.820132 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.820023    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-model-cache\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.820132 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.820061    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaf87a78-16ad-4867-a2a2-1552663eb835-tls-certs\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.820132 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.820093    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpmqk\" (UniqueName: \"kubernetes.io/projected/aaf87a78-16ad-4867-a2a2-1552663eb835-kube-api-access-hpmqk\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.827536 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.827507    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:16.921444 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.921406    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-dshm\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.921689 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.921452    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-tmp-dir\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.921689 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.921527    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-model-cache\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.921689 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.921554    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaf87a78-16ad-4867-a2a2-1552663eb835-tls-certs\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.921689 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.921583    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpmqk\" (UniqueName: \"kubernetes.io/projected/aaf87a78-16ad-4867-a2a2-1552663eb835-kube-api-access-hpmqk\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.921689 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.921671    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-home\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.922008 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.921978    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-model-cache\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.922068 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.922057    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-tmp-dir\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.922108 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.922096    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-home\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.923994 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.923974    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-dshm\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.924254 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.924238    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaf87a78-16ad-4867-a2a2-1552663eb835-tls-certs\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:16.948757 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:16.948726    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpmqk\" (UniqueName: \"kubernetes.io/projected/aaf87a78-16ad-4867-a2a2-1552663eb835-kube-api-access-hpmqk\") pod \"simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") " pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:17.051948 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:17.051861    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:17.122740 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:17.122709    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:17.365307 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:17.365267    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"]
Oct 30 06:36:17.367558 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:36:17.367529    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaaf87a78_16ad_4867_a2a2_1552663eb835.slice/crio-2ada24254a62da939484d912b660af9eadcc6fd64f25e9619c45da1aee728a61 WatchSource:0}: Error finding container 2ada24254a62da939484d912b660af9eadcc6fd64f25e9619c45da1aee728a61: Status 404 returned error can't find the container with id 2ada24254a62da939484d912b660af9eadcc6fd64f25e9619c45da1aee728a61
Oct 30 06:36:17.481958 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:17.481924    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl" event={"ID":"aaf87a78-16ad-4867-a2a2-1552663eb835","Type":"ContainerStarted","Data":"65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123"}
Oct 30 06:36:17.481958 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:17.481968    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl" event={"ID":"aaf87a78-16ad-4867-a2a2-1552663eb835","Type":"ContainerStarted","Data":"2ada24254a62da939484d912b660af9eadcc6fd64f25e9619c45da1aee728a61"}
Oct 30 06:36:17.533877 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:17.533820    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl" podStartSLOduration=1.533806244 podStartE2EDuration="1.533806244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:36:17.531838227 +0000 UTC m=+3881.277123560" watchObservedRunningTime="2026-10-30 06:36:17.533806244 +0000 UTC m=+3881.279091577"
Oct 30 06:36:18.052646 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:18.052609    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:18.834570 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:18.834535    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:20.441036 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:20.440998    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:21.968977 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:21.968942    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:23.438832 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:23.438795    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:25.033319 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:25.033288    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:26.767571 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:26.767519    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:27.052863 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:27.052775    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:27.065342 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:27.065311    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:27.544733 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:27.544700    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:36:28.194106 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:28.194072    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:29.659512 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:29.659480    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:31.275347 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:31.275310    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:32.551997 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:36:32.551963    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:36:32.761415 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:32.761378    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:34.248652 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:34.248614    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:35.360312 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:35.360279    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:35.864064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:35.864023    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:36.173749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.173712    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"]
Oct 30 06:36:36.177601 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.177574    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:36.198006 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.197982    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"]
Oct 30 06:36:36.208940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.208915    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 06:36:36.208940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.208914    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 06:36:36.210485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.210467    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 06:36:36.210614 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.210582    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:36:36.223470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.223437    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj4jn\" (UniqueName: \"kubernetes.io/projected/7f6acbe2-07b2-4529-9e84-238af4c53f19-kube-api-access-pj4jn\") pod \"maas-api-revoke-sub-88c0edca-4svrf\" (UID: \"7f6acbe2-07b2-4529-9e84-238af4c53f19\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:36.223617 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.223576    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-88c0edca-4svrf\" (UID: \"7f6acbe2-07b2-4529-9e84-238af4c53f19\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:36.223726 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.223660    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-xks-ca\") pod \"maas-api-revoke-sub-88c0edca-4svrf\" (UID: \"7f6acbe2-07b2-4529-9e84-238af4c53f19\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:36.307658 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.307628    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:36.325162 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.325129    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj4jn\" (UniqueName: \"kubernetes.io/projected/7f6acbe2-07b2-4529-9e84-238af4c53f19-kube-api-access-pj4jn\") pod \"maas-api-revoke-sub-88c0edca-4svrf\" (UID: \"7f6acbe2-07b2-4529-9e84-238af4c53f19\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:36.325260 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.325192    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-88c0edca-4svrf\" (UID: \"7f6acbe2-07b2-4529-9e84-238af4c53f19\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:36.325260 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.325216    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-xks-ca\") pod \"maas-api-revoke-sub-88c0edca-4svrf\" (UID: \"7f6acbe2-07b2-4529-9e84-238af4c53f19\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:36.325988 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.325969    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-88c0edca-4svrf\" (UID: \"7f6acbe2-07b2-4529-9e84-238af4c53f19\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:36.327782 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.327763    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-xks-ca\") pod \"maas-api-revoke-sub-88c0edca-4svrf\" (UID: \"7f6acbe2-07b2-4529-9e84-238af4c53f19\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:36.350457 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.350431    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj4jn\" (UniqueName: \"kubernetes.io/projected/7f6acbe2-07b2-4529-9e84-238af4c53f19-kube-api-access-pj4jn\") pod \"maas-api-revoke-sub-88c0edca-4svrf\" (UID: \"7f6acbe2-07b2-4529-9e84-238af4c53f19\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:36.507046 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.506974    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:36:36.512288 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.512264    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:36.812034 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:36.812007    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"]
Oct 30 06:36:36.814276 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:36:36.814249    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6acbe2_07b2_4529_9e84_238af4c53f19.slice/crio-34c3997a74d2cac79e8eb7e820bd0366c8f181cc2f90d95e3fac1d6cc09ee907 WatchSource:0}: Error finding container 34c3997a74d2cac79e8eb7e820bd0366c8f181cc2f90d95e3fac1d6cc09ee907: Status 404 returned error can't find the container with id 34c3997a74d2cac79e8eb7e820bd0366c8f181cc2f90d95e3fac1d6cc09ee907
Oct 30 06:36:37.012219 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:37.012196    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:36:37.033925 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:37.033904    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:36:37.436512 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:37.436481    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:37.592304 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:37.592264    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf" event={"ID":"7f6acbe2-07b2-4529-9e84-238af4c53f19","Type":"ContainerStarted","Data":"cdef0f13b4fa819dbc23f0a271c0f3b36168ff42af306984d2e03723e2cac4c6"}
Oct 30 06:36:37.592979 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:37.592321    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf" event={"ID":"7f6acbe2-07b2-4529-9e84-238af4c53f19","Type":"ContainerStarted","Data":"34c3997a74d2cac79e8eb7e820bd0366c8f181cc2f90d95e3fac1d6cc09ee907"}
Oct 30 06:36:37.637531 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:37.637470    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf" podStartSLOduration=1.637457068 podStartE2EDuration="1.637457068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:36:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:36:37.636305437 +0000 UTC m=+3901.381590768" watchObservedRunningTime="2026-10-30 06:36:37.637457068 +0000 UTC m=+3901.382742427"
Oct 30 06:36:38.599304 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:38.599267    5066 generic.go:356] "Generic (PLEG): container finished" podID="7f6acbe2-07b2-4529-9e84-238af4c53f19" containerID="cdef0f13b4fa819dbc23f0a271c0f3b36168ff42af306984d2e03723e2cac4c6" exitCode=0
Oct 30 06:36:38.599810 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:38.599314    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf" event={"ID":"7f6acbe2-07b2-4529-9e84-238af4c53f19","Type":"ContainerDied","Data":"cdef0f13b4fa819dbc23f0a271c0f3b36168ff42af306984d2e03723e2cac4c6"}
Oct 30 06:36:38.893428 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:38.893340    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:39.756050 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:39.756022    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:39.860692 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:39.860580    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f6acbe2-07b2-4529-9e84-238af4c53f19-kube-api-access-pj4jn\" (UniqueName: \"kubernetes.io/projected/7f6acbe2-07b2-4529-9e84-238af4c53f19-kube-api-access-pj4jn\") pod \"7f6acbe2-07b2-4529-9e84-238af4c53f19\" (UID: \"7f6acbe2-07b2-4529-9e84-238af4c53f19\") "
Oct 30 06:36:39.860692 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:39.860651    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-xks-ca\") pod \"7f6acbe2-07b2-4529-9e84-238af4c53f19\" (UID: \"7f6acbe2-07b2-4529-9e84-238af4c53f19\") "
Oct 30 06:36:39.860692 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:39.860691    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-openshift-ca\") pod \"7f6acbe2-07b2-4529-9e84-238af4c53f19\" (UID: \"7f6acbe2-07b2-4529-9e84-238af4c53f19\") "
Oct 30 06:36:39.861134 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:39.861103    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-openshift-ca" pod "7f6acbe2-07b2-4529-9e84-238af4c53f19" (UID: "7f6acbe2-07b2-4529-9e84-238af4c53f19"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:36:39.863025 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:39.862993    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-xks-ca" pod "7f6acbe2-07b2-4529-9e84-238af4c53f19" (UID: "7f6acbe2-07b2-4529-9e84-238af4c53f19"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:36:39.863111 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:39.863056    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f6acbe2-07b2-4529-9e84-238af4c53f19-kube-api-access-pj4jn" pod "7f6acbe2-07b2-4529-9e84-238af4c53f19" (UID: "7f6acbe2-07b2-4529-9e84-238af4c53f19"). InnerVolumeSpecName "kube-api-access-pj4jn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:36:39.962460 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:39.962419    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj4jn\" (UniqueName: \"kubernetes.io/projected/7f6acbe2-07b2-4529-9e84-238af4c53f19-kube-api-access-pj4jn\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:36:39.962460 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:39.962452    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:36:39.962460 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:39.962463    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7f6acbe2-07b2-4529-9e84-238af4c53f19-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:36:40.371015 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:40.370983    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:40.609994 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:40.609965    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"
Oct 30 06:36:40.610176 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:40.609997    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf" event={"ID":"7f6acbe2-07b2-4529-9e84-238af4c53f19","Type":"ContainerDied","Data":"34c3997a74d2cac79e8eb7e820bd0366c8f181cc2f90d95e3fac1d6cc09ee907"}
Oct 30 06:36:40.610176 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:40.610042    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34c3997a74d2cac79e8eb7e820bd0366c8f181cc2f90d95e3fac1d6cc09ee907"
Oct 30 06:36:41.968400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:41.968350    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:43.468990 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:43.468954    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:46.231909 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:46.231878    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:46.649566 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:46.649339    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:47.057878 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.057842    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:47.188359 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.188317    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"]
Oct 30 06:36:47.188849 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.188830    5066 cpu_manager.go:414] "containerMap: removing container" podUID="7f6acbe2-07b2-4529-9e84-238af4c53f19" containerName="revoke-keys"
Oct 30 06:36:47.188955 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.188849    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f6acbe2-07b2-4529-9e84-238af4c53f19" containerName="revoke-keys"
Oct 30 06:36:47.189049 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.189035    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f6acbe2-07b2-4529-9e84-238af4c53f19" containerName="revoke-keys"
Oct 30 06:36:47.191901 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.191878    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:47.212326 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.212298    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"]
Oct 30 06:36:47.223400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.223372    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 06:36:47.223559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.223403    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 06:36:47.223559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.223421    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 06:36:47.223559 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.223458    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:36:47.336881 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.336786    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9\" (UID: \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:47.337280 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.336950    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9\" (UID: \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:47.337280 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.336983    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knqcx\" (UniqueName: \"kubernetes.io/projected/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-kube-api-access-knqcx\") pod \"maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9\" (UID: \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:47.438011 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.437975    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9\" (UID: \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:47.438222 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.438063    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9\" (UID: \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:47.438222 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.438082    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knqcx\" (UniqueName: \"kubernetes.io/projected/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-kube-api-access-knqcx\") pod \"maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9\" (UID: \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:47.438780 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.438756    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9\" (UID: \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:47.440604 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.440567    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9\" (UID: \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:47.463298 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.463269    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knqcx\" (UniqueName: \"kubernetes.io/projected/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-kube-api-access-knqcx\") pod \"maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9\" (UID: \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:47.505729 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.505695    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:47.810542 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:47.810517    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"]
Oct 30 06:36:47.812156 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:36:47.812128    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c09a79_ba75_4849_bf2e_4df7ab1cc4f4.slice/crio-1c73dacc9989733ee8d1e3d74921c68c56aeab39d1fabe9eb20ec8d11f6f2e20 WatchSource:0}: Error finding container 1c73dacc9989733ee8d1e3d74921c68c56aeab39d1fabe9eb20ec8d11f6f2e20: Status 404 returned error can't find the container with id 1c73dacc9989733ee8d1e3d74921c68c56aeab39d1fabe9eb20ec8d11f6f2e20
Oct 30 06:36:48.472712 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:48.472677    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:48.651772 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:48.651736    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9" event={"ID":"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4","Type":"ContainerStarted","Data":"f6970ef81a56d9b6bc9d0d492159a5f3770f68617adf43f21fb6a1531f64ea9a"}
Oct 30 06:36:48.651912 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:48.651803    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9" event={"ID":"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4","Type":"ContainerStarted","Data":"1c73dacc9989733ee8d1e3d74921c68c56aeab39d1fabe9eb20ec8d11f6f2e20"}
Oct 30 06:36:49.657898 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:49.657864    5066 generic.go:356] "Generic (PLEG): container finished" podID="08c09a79-ba75-4849-bf2e-4df7ab1cc4f4" containerID="f6970ef81a56d9b6bc9d0d492159a5f3770f68617adf43f21fb6a1531f64ea9a" exitCode=0
Oct 30 06:36:49.658316 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:49.657945    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9" event={"ID":"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4","Type":"ContainerDied","Data":"f6970ef81a56d9b6bc9d0d492159a5f3770f68617adf43f21fb6a1531f64ea9a"}
Oct 30 06:36:49.816407 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:49.816382    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:49.929514 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:49.929424    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:49.960189 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:49.960158    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-xks-ca\") pod \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\" (UID: \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\") "
Oct 30 06:36:49.960384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:49.960224    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-openshift-ca\") pod \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\" (UID: \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\") "
Oct 30 06:36:49.960384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:49.960282    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-kube-api-access-knqcx\" (UniqueName: \"kubernetes.io/projected/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-kube-api-access-knqcx\") pod \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\" (UID: \"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4\") "
Oct 30 06:36:49.960621 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:49.960582    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-openshift-ca" pod "08c09a79-ba75-4849-bf2e-4df7ab1cc4f4" (UID: "08c09a79-ba75-4849-bf2e-4df7ab1cc4f4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:36:49.960771 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:49.960752    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:36:49.962601 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:49.962565    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-xks-ca" pod "08c09a79-ba75-4849-bf2e-4df7ab1cc4f4" (UID: "08c09a79-ba75-4849-bf2e-4df7ab1cc4f4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:36:49.962704 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:49.962639    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-kube-api-access-knqcx" pod "08c09a79-ba75-4849-bf2e-4df7ab1cc4f4" (UID: "08c09a79-ba75-4849-bf2e-4df7ab1cc4f4"). InnerVolumeSpecName "kube-api-access-knqcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:36:50.061290 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:50.061257    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knqcx\" (UniqueName: \"kubernetes.io/projected/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-kube-api-access-knqcx\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:36:50.061290 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:50.061285    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:36:50.664819 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:50.664729    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9" event={"ID":"08c09a79-ba75-4849-bf2e-4df7ab1cc4f4","Type":"ContainerDied","Data":"1c73dacc9989733ee8d1e3d74921c68c56aeab39d1fabe9eb20ec8d11f6f2e20"}
Oct 30 06:36:50.664819 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:50.664766    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"
Oct 30 06:36:50.665363 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:50.664774    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c73dacc9989733ee8d1e3d74921c68c56aeab39d1fabe9eb20ec8d11f6f2e20"
Oct 30 06:36:51.601441 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:51.601403    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:51.966866 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:51.966833    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"]
Oct 30 06:36:52.110256 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:52.110214    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:53.058227 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:53.058194    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:54.509419 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:54.509385    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:56.052485 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:56.052449    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:57.497963 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:57.497930    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:57.769804 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:57.769724    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:57.932979 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:57.932952    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:36:58.981457 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:36:58.981420    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:00.227568 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:00.227536    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:00.729401 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:00.729372    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:00.910920 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:00.910888    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"]
Oct 30 06:37:00.928365 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:00.928336    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-ee1c63-g7zn9"]
Oct 30 06:37:01.052063 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:01.051978    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:01.393108 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:01.393032    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:02.402886 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:02.402849    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08c09a79-ba75-4849-bf2e-4df7ab1cc4f4" path="/var/lib/kubelet/pods/08c09a79-ba75-4849-bf2e-4df7ab1cc4f4/volumes"
Oct 30 06:37:02.552098 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:37:02.552055    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:37:06.987743 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:06.987678    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl" podUID="aaf87a78-16ad-4867-a2a2-1552663eb835" containerName="main" containerID="cri-o://65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123" gracePeriod=45
Oct 30 06:37:07.253546 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.253486    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:37:07.312905 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.312872    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:07.330962 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.330929    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-tmp-dir\") pod \"aaf87a78-16ad-4867-a2a2-1552663eb835\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") "
Oct 30 06:37:07.331161 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.330985    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-dshm\") pod \"aaf87a78-16ad-4867-a2a2-1552663eb835\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") "
Oct 30 06:37:07.331161 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.331004    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aaf87a78-16ad-4867-a2a2-1552663eb835-tls-certs\" (UniqueName: \"kubernetes.io/secret/aaf87a78-16ad-4867-a2a2-1552663eb835-tls-certs\") pod \"aaf87a78-16ad-4867-a2a2-1552663eb835\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") "
Oct 30 06:37:07.331161 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.331040    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-home\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-home\") pod \"aaf87a78-16ad-4867-a2a2-1552663eb835\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") "
Oct 30 06:37:07.331161 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.331129    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aaf87a78-16ad-4867-a2a2-1552663eb835-kube-api-access-hpmqk\" (UniqueName: \"kubernetes.io/projected/aaf87a78-16ad-4867-a2a2-1552663eb835-kube-api-access-hpmqk\") pod \"aaf87a78-16ad-4867-a2a2-1552663eb835\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") "
Oct 30 06:37:07.331468 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.331164    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-model-cache\") pod \"aaf87a78-16ad-4867-a2a2-1552663eb835\" (UID: \"aaf87a78-16ad-4867-a2a2-1552663eb835\") "
Oct 30 06:37:07.331468 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.331290    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-tmp-dir" pod "aaf87a78-16ad-4867-a2a2-1552663eb835" (UID: "aaf87a78-16ad-4867-a2a2-1552663eb835"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:37:07.331607 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.331485    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-model-cache" pod "aaf87a78-16ad-4867-a2a2-1552663eb835" (UID: "aaf87a78-16ad-4867-a2a2-1552663eb835"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:37:07.331607 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.331561    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:37:07.331742 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.331643    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-home" pod "aaf87a78-16ad-4867-a2a2-1552663eb835" (UID: "aaf87a78-16ad-4867-a2a2-1552663eb835"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:37:07.333812 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.333777    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-dshm" pod "aaf87a78-16ad-4867-a2a2-1552663eb835" (UID: "aaf87a78-16ad-4867-a2a2-1552663eb835"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:37:07.333812 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.333801    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaf87a78-16ad-4867-a2a2-1552663eb835-kube-api-access-hpmqk" pod "aaf87a78-16ad-4867-a2a2-1552663eb835" (UID: "aaf87a78-16ad-4867-a2a2-1552663eb835"). InnerVolumeSpecName "kube-api-access-hpmqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:37:07.334009 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.333987    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaf87a78-16ad-4867-a2a2-1552663eb835-tls-certs" pod "aaf87a78-16ad-4867-a2a2-1552663eb835" (UID: "aaf87a78-16ad-4867-a2a2-1552663eb835"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:37:07.432948 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.432907    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:37:07.432948 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.432944    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaf87a78-16ad-4867-a2a2-1552663eb835-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:37:07.433232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.432958    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:37:07.433232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.432971    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpmqk\" (UniqueName: \"kubernetes.io/projected/aaf87a78-16ad-4867-a2a2-1552663eb835-kube-api-access-hpmqk\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:37:07.433232 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.432984    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaf87a78-16ad-4867-a2a2-1552663eb835-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:37:07.474904 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.474873    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:07.718917 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.718879    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:07.758969 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.758940    5066 generic.go:356] "Generic (PLEG): container finished" podID="aaf87a78-16ad-4867-a2a2-1552663eb835" containerID="65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123" exitCode=0
Oct 30 06:37:07.759166 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.758982    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl" event={"ID":"aaf87a78-16ad-4867-a2a2-1552663eb835","Type":"ContainerDied","Data":"65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123"}
Oct 30 06:37:07.759166 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.759044    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl" event={"ID":"aaf87a78-16ad-4867-a2a2-1552663eb835","Type":"ContainerDied","Data":"2ada24254a62da939484d912b660af9eadcc6fd64f25e9619c45da1aee728a61"}
Oct 30 06:37:07.759166 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.759064    5066 scope.go:122] "RemoveContainer" containerID="65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123"
Oct 30 06:37:07.759166 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.759001    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"
Oct 30 06:37:07.771405 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.771373    5066 scope.go:122] "RemoveContainer" containerID="65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123"
Oct 30 06:37:07.771801 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:37:07.771775    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123\": container with ID starting with 65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123 not found: ID does not exist" containerID="65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123"
Oct 30 06:37:07.771918 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.771813    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123"} err="failed to get container status \"65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123\": rpc error: code = NotFound desc = could not find container \"65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123\": container with ID starting with 65bcc92482526a687f5284d6aa7559e0a4a74e34dcedfef5704677bbd7e51123 not found: ID does not exist"
Oct 30 06:37:07.830637 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.830606    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"]
Oct 30 06:37:07.846789 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:07.846761    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-ee1c63/simulator-single-main-ee1c63-kserve-64c95c79d7-p6hjl"]
Oct 30 06:37:08.401402 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:08.401364    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aaf87a78-16ad-4867-a2a2-1552663eb835" path="/var/lib/kubelet/pods/aaf87a78-16ad-4867-a2a2-1552663eb835/volumes"
Oct 30 06:37:12.300664 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.300625    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"]
Oct 30 06:37:12.301298 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.301280    5066 cpu_manager.go:414] "containerMap: removing container" podUID="aaf87a78-16ad-4867-a2a2-1552663eb835" containerName="main"
Oct 30 06:37:12.301360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.301302    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aaf87a78-16ad-4867-a2a2-1552663eb835" containerName="main"
Oct 30 06:37:12.301360 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.301355    5066 cpu_manager.go:414] "containerMap: removing container" podUID="08c09a79-ba75-4849-bf2e-4df7ab1cc4f4" containerName="revoke-keys"
Oct 30 06:37:12.301446 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.301363    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08c09a79-ba75-4849-bf2e-4df7ab1cc4f4" containerName="revoke-keys"
Oct 30 06:37:12.301490 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.301467    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="08c09a79-ba75-4849-bf2e-4df7ab1cc4f4" containerName="revoke-keys"
Oct 30 06:37:12.301490 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.301487    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="aaf87a78-16ad-4867-a2a2-1552663eb835" containerName="main"
Oct 30 06:37:12.305069 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.305048    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.324042 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.324005    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"]
Oct 30 06:37:12.335852 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.335811    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 06:37:12.335852 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.335833    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2k6w7\""
Oct 30 06:37:12.338157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.338134    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 06:37:12.338157 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.338138    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-d358d1fd-kserve-self-signed-certs\""
Oct 30 06:37:12.381990 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.381949    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tmp-dir\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.382172 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.382018    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tls-certs\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.382172 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.382117    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b66fr\" (UniqueName: \"kubernetes.io/projected/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-kube-api-access-b66fr\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.382172 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.382151    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-model-cache\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.382281 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.382210    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-home\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.382323 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.382279    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-dshm\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.420107 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.420076    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:12.483832 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.483781    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b66fr\" (UniqueName: \"kubernetes.io/projected/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-kube-api-access-b66fr\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.484028 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.483873    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-model-cache\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.484028 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.483927    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-home\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.484121 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.484046    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-dshm\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.484121 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.484093    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tmp-dir\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.484189 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.484155    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tls-certs\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.485145 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.485109    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tmp-dir\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.485326 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.485189    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-model-cache\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.488092 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.485760    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-home\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.488092 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.487851    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-dshm\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.489044 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.489021    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tls-certs\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.510102 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.510070    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b66fr\" (UniqueName: \"kubernetes.io/projected/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-kube-api-access-b66fr\") pod \"e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") " pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.628547 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.628470    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:12.928848 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:12.928823    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"]
Oct 30 06:37:12.930498 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:37:12.930468    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f93e139_fc3e_4f8e_8b61_1b4e6c57b240.slice/crio-368f663d74757e66d4f0b38c045b8cc61cf1c7849aa013dc32878218cd4f0165 WatchSource:0}: Error finding container 368f663d74757e66d4f0b38c045b8cc61cf1c7849aa013dc32878218cd4f0165: Status 404 returned error can't find the container with id 368f663d74757e66d4f0b38c045b8cc61cf1c7849aa013dc32878218cd4f0165
Oct 30 06:37:13.794462 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:13.794422    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr" event={"ID":"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240","Type":"ContainerStarted","Data":"49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b"}
Oct 30 06:37:13.794462 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:13.794469    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr" event={"ID":"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240","Type":"ContainerStarted","Data":"368f663d74757e66d4f0b38c045b8cc61cf1c7849aa013dc32878218cd4f0165"}
Oct 30 06:37:13.842161 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:13.842112    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr" podStartSLOduration=1.8420984169999999 podStartE2EDuration="1.842098417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:37:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:37:13.840443898 +0000 UTC m=+3937.585729230" watchObservedRunningTime="2026-10-30 06:37:13.842098417 +0000 UTC m=+3937.587383749"
Oct 30 06:37:14.629019 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:14.628974    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:22.629183 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:22.629128    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:22.641832 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:22.641807    5066 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:22.858071 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:22.858044    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:37:28.188300 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:28.188271    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:29.551539 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:29.551477    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:29.765662 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:29.765629    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:32.558840 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:37:32.558803    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:37:34.442747 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:34.442700    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-n57mc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.185:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 06:37:34.443400 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:34.443368    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc" podUID="74763aed-8beb-4048-8d84-7e52fa1c19dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.185:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 06:37:34.506783 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:34.506731    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-qgv7r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.186:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 06:37:34.506986 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:34.506811    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r" podUID="4d54096d-00ef-4c39-a0e8-6d1abf5f6640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.186:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 06:37:35.480379 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:35.480347    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:36.421504 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:36.421473    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:42.909166 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:42.909122    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:45.500473 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:45.500443    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:49.454264 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:49.453706    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-n57mc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.185:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 06:37:49.454264 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:49.453824    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-n57mc" podUID="74763aed-8beb-4048-8d84-7e52fa1c19dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.185:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 06:37:49.514763 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:49.514714    5066 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-qgv7r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.186:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 06:37:49.514940 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:49.514793    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-qgv7r" podUID="4d54096d-00ef-4c39-a0e8-6d1abf5f6640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.186:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 06:37:56.765914 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:56.765872    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"]
Oct 30 06:37:56.872973 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:56.872774    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:37:59.111833 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:37:59.111800    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:38:00.015215 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:00.015181    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:38:00.180501 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:00.180445    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:38:02.551536 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:38:02.551499    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:38:11.032067 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:11.032025    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"]
Oct 30 06:38:11.046115 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:11.046077    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6a85919a-zkgkq"]
Oct 30 06:38:11.787973 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:11.787905    5066 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr" podUID="7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" containerName="main" containerID="cri-o://49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b" gracePeriod=45
Oct 30 06:38:11.847958 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:11.847911    5066 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr" podUID="7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" containerName="main" probeResult="failure" output="Get \"https://10.132.0.195:8000/ready\": dial tcp 10.132.0.195:8000: connect: connection refused"
Oct 30 06:38:12.056942 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.056912    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:38:12.152733 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.152692    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-home\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-home\") pod \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") "
Oct 30 06:38:12.152900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.152750    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-kube-api-access-b66fr\" (UniqueName: \"kubernetes.io/projected/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-kube-api-access-b66fr\") pod \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") "
Oct 30 06:38:12.152900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.152804    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-dshm\") pod \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") "
Oct 30 06:38:12.152900 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.152870    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-model-cache\") pod \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") "
Oct 30 06:38:12.153064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.152903    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tmp-dir\") pod \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") "
Oct 30 06:38:12.153064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.152930    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tls-certs\" (UniqueName: \"kubernetes.io/secret/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tls-certs\") pod \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\" (UID: \"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240\") "
Oct 30 06:38:12.153064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.153033    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-home" pod "7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" (UID: "7f93e139-fc3e-4f8e-8b61-1b4e6c57b240"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:38:12.153258 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.153221    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tmp-dir" pod "7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" (UID: "7f93e139-fc3e-4f8e-8b61-1b4e6c57b240"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:38:12.153258 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.153238    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-model-cache" pod "7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" (UID: "7f93e139-fc3e-4f8e-8b61-1b4e6c57b240"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:38:12.153258 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.153239    5066 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-home\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:38:12.155304 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.155272    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-dshm" pod "7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" (UID: "7f93e139-fc3e-4f8e-8b61-1b4e6c57b240"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 06:38:12.155422 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.155364    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tls-certs" pod "7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" (UID: "7f93e139-fc3e-4f8e-8b61-1b4e6c57b240"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:38:12.155422 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.155369    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-kube-api-access-b66fr" pod "7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" (UID: "7f93e139-fc3e-4f8e-8b61-1b4e6c57b240"). InnerVolumeSpecName "kube-api-access-b66fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:38:12.161519 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.161493    5066 generic.go:356] "Generic (PLEG): container finished" podID="7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" containerID="49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b" exitCode=0
Oct 30 06:38:12.161760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.161543    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"
Oct 30 06:38:12.161760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.161580    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr" event={"ID":"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240","Type":"ContainerDied","Data":"49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b"}
Oct 30 06:38:12.161760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.161661    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr" event={"ID":"7f93e139-fc3e-4f8e-8b61-1b4e6c57b240","Type":"ContainerDied","Data":"368f663d74757e66d4f0b38c045b8cc61cf1c7849aa013dc32878218cd4f0165"}
Oct 30 06:38:12.161760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.161685    5066 scope.go:122] "RemoveContainer" containerID="49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b"
Oct 30 06:38:12.178200 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.178179    5066 scope.go:122] "RemoveContainer" containerID="49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b"
Oct 30 06:38:12.178624 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:38:12.178549    5066 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b\": container with ID starting with 49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b not found: ID does not exist" containerID="49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b"
Oct 30 06:38:12.178624 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.178581    5066 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b"} err="failed to get container status \"49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b\": rpc error: code = NotFound desc = could not find container \"49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b\": container with ID starting with 49a6128025b19283bc58f070006e7e131a7292c980391345d033c3b7d02e4a6b not found: ID does not exist"
Oct 30 06:38:12.218938 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.218904    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"]
Oct 30 06:38:12.223192 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.223161    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-d358d1fd-kserve-78bd798dc7-pnjwr"]
Oct 30 06:38:12.254247 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.254202    5066 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-model-cache\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:38:12.254247 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.254234    5066 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tmp-dir\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:38:12.254247 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.254242    5066 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-tls-certs\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:38:12.254247 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.254252    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b66fr\" (UniqueName: \"kubernetes.io/projected/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-kube-api-access-b66fr\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:38:12.254247 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.254261    5066 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240-dshm\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:38:12.401332 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.401244    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" path="/var/lib/kubelet/pods/7f93e139-fc3e-4f8e-8b61-1b4e6c57b240/volumes"
Oct 30 06:38:12.401704 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:12.401690    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5ea76c7-2c83-4499-8b15-21d7c16d3727" path="/var/lib/kubelet/pods/b5ea76c7-2c83-4499-8b15-21d7c16d3727/volumes"
Oct 30 06:38:32.552391 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:38:32.552358    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:38:35.617207 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:35.617178    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:38:36.551371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:38:36.551340    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:39:02.341471 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.341435    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"]
Oct 30 06:39:02.342217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.341884    5066 cpu_manager.go:414] "containerMap: removing container" podUID="7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" containerName="main"
Oct 30 06:39:02.342217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.341895    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" containerName="main"
Oct 30 06:39:02.342217 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.341984    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f93e139-fc3e-4f8e-8b61-1b4e6c57b240" containerName="main"
Oct 30 06:39:02.345194 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.345178    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:02.366059 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.366032    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"]
Oct 30 06:39:02.377823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.377797    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 06:39:02.377823 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.377802    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 06:39:02.378026 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.377802    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 06:39:02.378026 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.377864    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:39:02.446480 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.446441    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-mmvdq\" (UID: \"bcca4f41-f10e-4f72-81a9-c4b500004845\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:02.446700 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.446502    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddqwp\" (UniqueName: \"kubernetes.io/projected/bcca4f41-f10e-4f72-81a9-c4b500004845-kube-api-access-ddqwp\") pod \"maas-api-revoke-sub-f03629db-mmvdq\" (UID: \"bcca4f41-f10e-4f72-81a9-c4b500004845\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:02.446775 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.446725    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-mmvdq\" (UID: \"bcca4f41-f10e-4f72-81a9-c4b500004845\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:02.547368 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.547333    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-mmvdq\" (UID: \"bcca4f41-f10e-4f72-81a9-c4b500004845\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:02.547569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.547418    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-mmvdq\" (UID: \"bcca4f41-f10e-4f72-81a9-c4b500004845\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:02.547569 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.547487    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddqwp\" (UniqueName: \"kubernetes.io/projected/bcca4f41-f10e-4f72-81a9-c4b500004845-kube-api-access-ddqwp\") pod \"maas-api-revoke-sub-f03629db-mmvdq\" (UID: \"bcca4f41-f10e-4f72-81a9-c4b500004845\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:02.548531 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.548360    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-mmvdq\" (UID: \"bcca4f41-f10e-4f72-81a9-c4b500004845\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:02.550632 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.550609    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-mmvdq\" (UID: \"bcca4f41-f10e-4f72-81a9-c4b500004845\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:02.550938 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:39:02.550915    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:39:02.580832 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.580799    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddqwp\" (UniqueName: \"kubernetes.io/projected/bcca4f41-f10e-4f72-81a9-c4b500004845-kube-api-access-ddqwp\") pod \"maas-api-revoke-sub-f03629db-mmvdq\" (UID: \"bcca4f41-f10e-4f72-81a9-c4b500004845\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:02.657839 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.657814    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:02.966482 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:02.966454    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"]
Oct 30 06:39:02.968065 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:39:02.968036    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcca4f41_f10e_4f72_81a9_c4b500004845.slice/crio-2882e687fb4a4918f060e034da0fd28aa4664bde7dff3f4ef3b82ed11039ccb5 WatchSource:0}: Error finding container 2882e687fb4a4918f060e034da0fd28aa4664bde7dff3f4ef3b82ed11039ccb5: Status 404 returned error can't find the container with id 2882e687fb4a4918f060e034da0fd28aa4664bde7dff3f4ef3b82ed11039ccb5
Oct 30 06:39:03.516927 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:03.516842    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq" event={"ID":"bcca4f41-f10e-4f72-81a9-c4b500004845","Type":"ContainerStarted","Data":"dd2a1f32b17ffc2a203a645a943cfb41933c2a6ae308cd970d61defd60a5ede6"}
Oct 30 06:39:03.516927 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:03.516896    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq" event={"ID":"bcca4f41-f10e-4f72-81a9-c4b500004845","Type":"ContainerStarted","Data":"2882e687fb4a4918f060e034da0fd28aa4664bde7dff3f4ef3b82ed11039ccb5"}
Oct 30 06:39:03.565630 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:03.565566    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq" podStartSLOduration=1.56555193 podStartE2EDuration="1.56555193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:39:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:39:03.564574895 +0000 UTC m=+4047.309860209" watchObservedRunningTime="2026-10-30 06:39:03.56555193 +0000 UTC m=+4047.310837262"
Oct 30 06:39:04.523072 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:04.523037    5066 generic.go:356] "Generic (PLEG): container finished" podID="bcca4f41-f10e-4f72-81a9-c4b500004845" containerID="dd2a1f32b17ffc2a203a645a943cfb41933c2a6ae308cd970d61defd60a5ede6" exitCode=0
Oct 30 06:39:04.523479 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:04.523077    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq" event={"ID":"bcca4f41-f10e-4f72-81a9-c4b500004845","Type":"ContainerDied","Data":"dd2a1f32b17ffc2a203a645a943cfb41933c2a6ae308cd970d61defd60a5ede6"}
Oct 30 06:39:05.672525 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:05.672500    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:05.779792 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:05.779704    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-xks-ca\") pod \"bcca4f41-f10e-4f72-81a9-c4b500004845\" (UID: \"bcca4f41-f10e-4f72-81a9-c4b500004845\") "
Oct 30 06:39:05.779953 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:05.779831    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-openshift-ca\") pod \"bcca4f41-f10e-4f72-81a9-c4b500004845\" (UID: \"bcca4f41-f10e-4f72-81a9-c4b500004845\") "
Oct 30 06:39:05.779953 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:05.779896    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bcca4f41-f10e-4f72-81a9-c4b500004845-kube-api-access-ddqwp\" (UniqueName: \"kubernetes.io/projected/bcca4f41-f10e-4f72-81a9-c4b500004845-kube-api-access-ddqwp\") pod \"bcca4f41-f10e-4f72-81a9-c4b500004845\" (UID: \"bcca4f41-f10e-4f72-81a9-c4b500004845\") "
Oct 30 06:39:05.780275 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:05.780236    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-openshift-ca" pod "bcca4f41-f10e-4f72-81a9-c4b500004845" (UID: "bcca4f41-f10e-4f72-81a9-c4b500004845"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:39:05.781934 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:05.781913    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-xks-ca" pod "bcca4f41-f10e-4f72-81a9-c4b500004845" (UID: "bcca4f41-f10e-4f72-81a9-c4b500004845"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:39:05.782220 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:05.782198    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcca4f41-f10e-4f72-81a9-c4b500004845-kube-api-access-ddqwp" pod "bcca4f41-f10e-4f72-81a9-c4b500004845" (UID: "bcca4f41-f10e-4f72-81a9-c4b500004845"). InnerVolumeSpecName "kube-api-access-ddqwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:39:05.881187 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:05.881153    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddqwp\" (UniqueName: \"kubernetes.io/projected/bcca4f41-f10e-4f72-81a9-c4b500004845-kube-api-access-ddqwp\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:39:05.881187 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:05.881182    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:39:05.881187 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:05.881192    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/bcca4f41-f10e-4f72-81a9-c4b500004845-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:39:06.534387 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:06.534345    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq" event={"ID":"bcca4f41-f10e-4f72-81a9-c4b500004845","Type":"ContainerDied","Data":"2882e687fb4a4918f060e034da0fd28aa4664bde7dff3f4ef3b82ed11039ccb5"}
Oct 30 06:39:06.534564 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:06.534403    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2882e687fb4a4918f060e034da0fd28aa4664bde7dff3f4ef3b82ed11039ccb5"
Oct 30 06:39:06.534564 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:06.534368    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-mmvdq"
Oct 30 06:39:14.853864 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:14.853830    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"]
Oct 30 06:39:14.854292 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:14.854278    5066 cpu_manager.go:414] "containerMap: removing container" podUID="bcca4f41-f10e-4f72-81a9-c4b500004845" containerName="revoke-keys"
Oct 30 06:39:14.854335 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:14.854294    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bcca4f41-f10e-4f72-81a9-c4b500004845" containerName="revoke-keys"
Oct 30 06:39:14.854371 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:14.854360    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="bcca4f41-f10e-4f72-81a9-c4b500004845" containerName="revoke-keys"
Oct 30 06:39:14.857478 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:14.857461    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:14.876281 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:14.876237    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"]
Oct 30 06:39:14.888193 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:14.888160    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:39:14.888193 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:14.888168    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 06:39:14.888960 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:14.888939    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 06:39:14.889086 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:14.888980    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 06:39:15.002748 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:15.002712    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs7sg\" (UniqueName: \"kubernetes.io/projected/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-kube-api-access-xs7sg\") pod \"maas-api-revoke-sub-0165d7d2-ssltj\" (UID: \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:15.002748 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:15.002746    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-ssltj\" (UID: \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:15.003015 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:15.002969    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-ssltj\" (UID: \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:15.104168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:15.104079    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-ssltj\" (UID: \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:15.104168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:15.104122    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs7sg\" (UniqueName: \"kubernetes.io/projected/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-kube-api-access-xs7sg\") pod \"maas-api-revoke-sub-0165d7d2-ssltj\" (UID: \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:15.104168 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:15.104145    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-ssltj\" (UID: \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:15.104784 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:15.104764    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-ssltj\" (UID: \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:15.106704 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:15.106686    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-ssltj\" (UID: \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:15.129600 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:15.129555    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs7sg\" (UniqueName: \"kubernetes.io/projected/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-kube-api-access-xs7sg\") pod \"maas-api-revoke-sub-0165d7d2-ssltj\" (UID: \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:15.170828 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:15.170803    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:15.470146 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:15.470103    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"]
Oct 30 06:39:15.471839 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:39:15.471814    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda88dd317_9c8d_4829_acbb_c0ae7e5d1d76.slice/crio-a1615710fa09e8562fd19fee4a65f52c50d4d07449d6c91c0dc203a959322794 WatchSource:0}: Error finding container a1615710fa09e8562fd19fee4a65f52c50d4d07449d6c91c0dc203a959322794: Status 404 returned error can't find the container with id a1615710fa09e8562fd19fee4a65f52c50d4d07449d6c91c0dc203a959322794
Oct 30 06:39:15.616766 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:15.616724    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj" event={"ID":"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76","Type":"ContainerStarted","Data":"a1615710fa09e8562fd19fee4a65f52c50d4d07449d6c91c0dc203a959322794"}
Oct 30 06:39:16.623753 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:16.623660    5066 generic.go:356] "Generic (PLEG): container finished" podID="a88dd317-9c8d-4829-acbb-c0ae7e5d1d76" containerID="d9c371f2b9d942de140f566f1fea1f6dd70c4e6f6a41db72156213f7d388e1f2" exitCode=0
Oct 30 06:39:16.623753 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:16.623747    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj" event={"ID":"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76","Type":"ContainerDied","Data":"d9c371f2b9d942de140f566f1fea1f6dd70c4e6f6a41db72156213f7d388e1f2"}
Oct 30 06:39:17.769974 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:17.769950    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:17.932192 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:17.932162    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-xks-ca\") pod \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\" (UID: \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\") "
Oct 30 06:39:17.932410 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:17.932256    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-kube-api-access-xs7sg\" (UniqueName: \"kubernetes.io/projected/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-kube-api-access-xs7sg\") pod \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\" (UID: \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\") "
Oct 30 06:39:17.932410 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:17.932287    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-openshift-ca\") pod \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\" (UID: \"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76\") "
Oct 30 06:39:17.932705 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:17.932682    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-openshift-ca" pod "a88dd317-9c8d-4829-acbb-c0ae7e5d1d76" (UID: "a88dd317-9c8d-4829-acbb-c0ae7e5d1d76"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:39:17.934502 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:17.934481    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-xks-ca" pod "a88dd317-9c8d-4829-acbb-c0ae7e5d1d76" (UID: "a88dd317-9c8d-4829-acbb-c0ae7e5d1d76"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:39:17.934576 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:17.934561    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-kube-api-access-xs7sg" pod "a88dd317-9c8d-4829-acbb-c0ae7e5d1d76" (UID: "a88dd317-9c8d-4829-acbb-c0ae7e5d1d76"). InnerVolumeSpecName "kube-api-access-xs7sg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:39:18.033993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:18.033957    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xs7sg\" (UniqueName: \"kubernetes.io/projected/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-kube-api-access-xs7sg\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:39:18.033993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:18.033986    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:39:18.033993 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:18.033997    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a88dd317-9c8d-4829-acbb-c0ae7e5d1d76-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:39:18.635055 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:18.635025    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj"
Oct 30 06:39:18.635239 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:18.635062    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-ssltj" event={"ID":"a88dd317-9c8d-4829-acbb-c0ae7e5d1d76","Type":"ContainerDied","Data":"a1615710fa09e8562fd19fee4a65f52c50d4d07449d6c91c0dc203a959322794"}
Oct 30 06:39:18.635239 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:18.635102    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1615710fa09e8562fd19fee4a65f52c50d4d07449d6c91c0dc203a959322794"
Oct 30 06:39:25.831234 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.831199    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"]
Oct 30 06:39:25.831664 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.831651    5066 cpu_manager.go:414] "containerMap: removing container" podUID="a88dd317-9c8d-4829-acbb-c0ae7e5d1d76" containerName="revoke-keys"
Oct 30 06:39:25.831720 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.831666    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a88dd317-9c8d-4829-acbb-c0ae7e5d1d76" containerName="revoke-keys"
Oct 30 06:39:25.831777 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.831767    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="a88dd317-9c8d-4829-acbb-c0ae7e5d1d76" containerName="revoke-keys"
Oct 30 06:39:25.835058 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.835035    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:25.860522 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.860488    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"]
Oct 30 06:39:25.870459 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.870430    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 06:39:25.870459 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.870451    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7z5tn\""
Oct 30 06:39:25.870702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.870458    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 06:39:25.870702 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.870470    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 06:39:25.904462 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.904432    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-wttpl\" (UID: \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:25.904663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.904481    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5l4m\" (UniqueName: \"kubernetes.io/projected/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-kube-api-access-d5l4m\") pod \"maas-api-revoke-sub-6b19338f-wttpl\" (UID: \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:25.904663 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:25.904506    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-wttpl\" (UID: \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:26.005805 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:26.005764    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-wttpl\" (UID: \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:26.006027 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:26.005819    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5l4m\" (UniqueName: \"kubernetes.io/projected/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-kube-api-access-d5l4m\") pod \"maas-api-revoke-sub-6b19338f-wttpl\" (UID: \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:26.006027 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:26.005844    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-wttpl\" (UID: \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:26.006495 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:26.006475    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-wttpl\" (UID: \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:26.008343 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:26.008323    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-wttpl\" (UID: \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:26.033956 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:26.033930    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5l4m\" (UniqueName: \"kubernetes.io/projected/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-kube-api-access-d5l4m\") pod \"maas-api-revoke-sub-6b19338f-wttpl\" (UID: \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:26.152850 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:26.152766    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:26.451867 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:26.451834    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"]
Oct 30 06:39:26.453323 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:39:26.453296    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee37a413_7b18_4bc9_8ac1_aae686d51fc4.slice/crio-dedbca61f6a03c8f94ff364c0fafe4d8cd489c6c12795d9bedd9738d9b323e36 WatchSource:0}: Error finding container dedbca61f6a03c8f94ff364c0fafe4d8cd489c6c12795d9bedd9738d9b323e36: Status 404 returned error can't find the container with id dedbca61f6a03c8f94ff364c0fafe4d8cd489c6c12795d9bedd9738d9b323e36
Oct 30 06:39:26.677399 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:26.677366    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl" event={"ID":"ee37a413-7b18-4bc9-8ac1-aae686d51fc4","Type":"ContainerStarted","Data":"dedbca61f6a03c8f94ff364c0fafe4d8cd489c6c12795d9bedd9738d9b323e36"}
Oct 30 06:39:27.684160 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:27.684128    5066 generic.go:356] "Generic (PLEG): container finished" podID="ee37a413-7b18-4bc9-8ac1-aae686d51fc4" containerID="31b651e090e7f406fb5f10a9effe34dcd545a2f9cc046decf7ef203e30cfcf5d" exitCode=0
Oct 30 06:39:27.684548 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:27.684172    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl" event={"ID":"ee37a413-7b18-4bc9-8ac1-aae686d51fc4","Type":"ContainerDied","Data":"31b651e090e7f406fb5f10a9effe34dcd545a2f9cc046decf7ef203e30cfcf5d"}
Oct 30 06:39:28.836299 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:28.836267    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:28.933340 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:28.933305    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-openshift-ca\") pod \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\" (UID: \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\") "
Oct 30 06:39:28.933524 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:28.933355    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-xks-ca\") pod \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\" (UID: \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\") "
Oct 30 06:39:28.933524 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:28.933376    5066 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-kube-api-access-d5l4m\" (UniqueName: \"kubernetes.io/projected/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-kube-api-access-d5l4m\") pod \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\" (UID: \"ee37a413-7b18-4bc9-8ac1-aae686d51fc4\") "
Oct 30 06:39:28.933734 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:28.933710    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-openshift-ca" pod "ee37a413-7b18-4bc9-8ac1-aae686d51fc4" (UID: "ee37a413-7b18-4bc9-8ac1-aae686d51fc4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 06:39:28.935665 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:28.935639    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-kube-api-access-d5l4m" pod "ee37a413-7b18-4bc9-8ac1-aae686d51fc4" (UID: "ee37a413-7b18-4bc9-8ac1-aae686d51fc4"). InnerVolumeSpecName "kube-api-access-d5l4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 06:39:28.935818 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:28.935700    5066 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-xks-ca" pod "ee37a413-7b18-4bc9-8ac1-aae686d51fc4" (UID: "ee37a413-7b18-4bc9-8ac1-aae686d51fc4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 06:39:29.034452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:29.034416    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-xks-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:39:29.034452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:29.034444    5066 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5l4m\" (UniqueName: \"kubernetes.io/projected/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-kube-api-access-d5l4m\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:39:29.034452 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:29.034454    5066 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ee37a413-7b18-4bc9-8ac1-aae686d51fc4-maas-api-openshift-ca\") on node \"ip-10-0-136-173.ec2.internal\" DevicePath \"\""
Oct 30 06:39:29.703200 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:29.703165    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl" event={"ID":"ee37a413-7b18-4bc9-8ac1-aae686d51fc4","Type":"ContainerDied","Data":"dedbca61f6a03c8f94ff364c0fafe4d8cd489c6c12795d9bedd9738d9b323e36"}
Oct 30 06:39:29.703358 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:29.703207    5066 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wttpl"
Oct 30 06:39:29.703358 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:29.703214    5066 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dedbca61f6a03c8f94ff364c0fafe4d8cd489c6c12795d9bedd9738d9b323e36"
Oct 30 06:39:30.187348 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:30.187308    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:39:32.551964 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:39:32.551925    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:39:35.733896 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:35.733862    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:39:36.672384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:36.672332    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:39:38.309449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:38.309413    5066 scope.go:122] "RemoveContainer" containerID="ab350ab9ab0b904fa8ad9e09744e107636f63c9c4fff703814849ddabf23fa8d"
Oct 30 06:39:38.318808 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:38.318789    5066 scope.go:122] "RemoveContainer" containerID="3751c1650429c026a4c80069aea882963d1fc46c62104f7e373af3a08b11c5b3"
Oct 30 06:39:47.902487 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:47.902401    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-mz8np_18075ec4-0c86-4b95-9cf7-20c74ed041ff/manager/0.log"
Oct 30 06:39:48.305608 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:48.305551    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-hbjqb_ea7416de-1581-4482-b9c8-3615dc434510/manager/0.log"
Oct 30 06:39:48.803561 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:48.803531    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-ltdvw_e7a2529b-780f-44d4-9958-3791774aa7a5/server/0.log"
Oct 30 06:39:48.997219 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:48.997183    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-f87fcf56c-nbt6q_730f3ec9-a3a7-4d99-8064-ee403ca67a48/manager/0.log"
Oct 30 06:39:49.452563 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:49.452535    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f7d897f7-4c5zt_26dbd02d-2395-4eec-b0d0-d46ba647d76f/copy-manifests/0.log"
Oct 30 06:39:49.482426 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:49.482398    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f7d897f7-4c5zt_26dbd02d-2395-4eec-b0d0-d46ba647d76f/manager/0.log"
Oct 30 06:39:50.434906 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:50.434875    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-nlrqh_1c5632fa-9bfe-424f-a58f-0cb02928fc63/proxy/0.log"
Oct 30 06:39:51.617384 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:51.617350    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-ssltj_a88dd317-9c8d-4829-acbb-c0ae7e5d1d76/revoke-keys/0.log"
Oct 30 06:39:52.369807 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:52.369778    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-wttpl_ee37a413-7b18-4bc9-8ac1-aae686d51fc4/revoke-keys/0.log"
Oct 30 06:39:52.555603 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:52.555553    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-88c0edca-4svrf_7f6acbe2-07b2-4529-9e84-238af4c53f19/revoke-keys/0.log"
Oct 30 06:39:53.090354 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:53.090326    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-mmvdq_bcca4f41-f10e-4f72-81a9-c4b500004845/revoke-keys/0.log"
Oct 30 06:39:55.026144 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:55.026108    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-4lplv_d25dce9a-0499-413d-99c7-0766aaf63fa1/manager/0.log"
Oct 30 06:39:55.223253 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:55.223211    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-6k4sq_3692525c-1db2-4a7a-ba04-c26bdcf52282/manager/0.log"
Oct 30 06:39:56.093288 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:56.093252    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:39:56.259573 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:56.259532    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:39:56.941470 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:56.941427    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2_e1bf555f-5bd6-4b63-8c64-04d79c058cb4/istio-proxy/0.log"
Oct 30 06:39:57.712258 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:57.712215    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-5cp4f_7010a90d-ab6c-4c3c-a5af-f04fea3e6caa/istio-proxy/0.log"
Oct 30 06:39:58.724140 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:58.724100    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-n57mc_74763aed-8beb-4048-8d84-7e52fa1c19dd/istio-proxy/0.log"
Oct 30 06:39:59.131281 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:39:59.131190    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-qgv7r_4d54096d-00ef-4c39-a0e8-6d1abf5f6640/istio-proxy/0.log"
Oct 30 06:40:00.200514 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:00.200480    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59857694c5-wq9fj_4684e0e6-736f-41bf-8f22-77245e339d58/router/0.log"
Oct 30 06:40:02.551882 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:40:02.551847    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:40:13.797309 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:13.797278    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-mz8np_18075ec4-0c86-4b95-9cf7-20c74ed041ff/manager/0.log"
Oct 30 06:40:14.188533 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:14.188502    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-hbjqb_ea7416de-1581-4482-b9c8-3615dc434510/manager/0.log"
Oct 30 06:40:14.639499 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:14.639409    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-ltdvw_e7a2529b-780f-44d4-9958-3791774aa7a5/server/0.log"
Oct 30 06:40:14.827081 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:14.827043    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-f87fcf56c-nbt6q_730f3ec9-a3a7-4d99-8064-ee403ca67a48/manager/0.log"
Oct 30 06:40:15.284837 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:15.284811    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f7d897f7-4c5zt_26dbd02d-2395-4eec-b0d0-d46ba647d76f/copy-manifests/0.log"
Oct 30 06:40:15.313505 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:15.313474    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f7d897f7-4c5zt_26dbd02d-2395-4eec-b0d0-d46ba647d76f/manager/0.log"
Oct 30 06:40:15.950310 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:15.950281    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-nlrqh_1c5632fa-9bfe-424f-a58f-0cb02928fc63/proxy/0.log"
Oct 30 06:40:16.886872 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:16.886835    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-ssltj_a88dd317-9c8d-4829-acbb-c0ae7e5d1d76/revoke-keys/0.log"
Oct 30 06:40:17.607467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:17.607440    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-wttpl_ee37a413-7b18-4bc9-8ac1-aae686d51fc4/revoke-keys/0.log"
Oct 30 06:40:17.777749 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:17.777719    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-88c0edca-4svrf_7f6acbe2-07b2-4529-9e84-238af4c53f19/revoke-keys/0.log"
Oct 30 06:40:18.310731 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:18.310703    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-mmvdq_bcca4f41-f10e-4f72-81a9-c4b500004845/revoke-keys/0.log"
Oct 30 06:40:19.845866 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:19.845835    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-4lplv_d25dce9a-0499-413d-99c7-0766aaf63fa1/manager/0.log"
Oct 30 06:40:20.041669 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:20.041634    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-6k4sq_3692525c-1db2-4a7a-ba04-c26bdcf52282/manager/0.log"
Oct 30 06:40:21.411439 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:21.411409    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2_e1bf555f-5bd6-4b63-8c64-04d79c058cb4/istio-proxy/0.log"
Oct 30 06:40:22.182873 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:22.182830    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-5cp4f_7010a90d-ab6c-4c3c-a5af-f04fea3e6caa/istio-proxy/0.log"
Oct 30 06:40:23.181755 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:23.181711    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-n57mc_74763aed-8beb-4048-8d84-7e52fa1c19dd/istio-proxy/0.log"
Oct 30 06:40:23.552244 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:23.552143    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-qgv7r_4d54096d-00ef-4c39-a0e8-6d1abf5f6640/istio-proxy/0.log"
Oct 30 06:40:24.523429 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:24.523400    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59857694c5-wq9fj_4684e0e6-736f-41bf-8f22-77245e339d58/router/0.log"
Oct 30 06:40:32.551301 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:40:32.551268    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:40:35.884053 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:35.884023    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:40:36.797053 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:40:36.797014    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:41:02.552348 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:41:02.552308    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:41:32.551309 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:41:32.551264    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:41:35.985064 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:35.985021    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:41:36.928760 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:36.928724    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:41:37.043607 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.043570    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:41:37.066514 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.066488    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:41:37.274056 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.273973    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-88m2b/must-gather-wtt2q"]
Oct 30 06:41:37.274477 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.274461    5066 cpu_manager.go:414] "containerMap: removing container" podUID="ee37a413-7b18-4bc9-8ac1-aae686d51fc4" containerName="revoke-keys"
Oct 30 06:41:37.274566 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.274478    5066 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee37a413-7b18-4bc9-8ac1-aae686d51fc4" containerName="revoke-keys"
Oct 30 06:41:37.274566 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.274564    5066 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee37a413-7b18-4bc9-8ac1-aae686d51fc4" containerName="revoke-keys"
Oct 30 06:41:37.277902 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.277887    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-88m2b/must-gather-wtt2q"
Oct 30 06:41:37.294976 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.294955    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-88m2b\"/\"openshift-service-ca.crt\""
Oct 30 06:41:37.298869 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.298845    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:41:37.309287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.309262    5066 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-88m2b\"/\"kube-root-ca.crt\""
Oct 30 06:41:37.309287 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.309271    5066 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-88m2b\"/\"default-dockercfg-hccqc\""
Oct 30 06:41:37.311424 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.311399    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-88m2b/must-gather-wtt2q"]
Oct 30 06:41:37.370112 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.370077    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/528b3ff9-2053-47e6-89f9-cc0b95774b4b-must-gather-output\") pod \"must-gather-wtt2q\" (UID: \"528b3ff9-2053-47e6-89f9-cc0b95774b4b\") " pod="openshift-must-gather-88m2b/must-gather-wtt2q"
Oct 30 06:41:37.370305 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.370214    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-746f2\" (UniqueName: \"kubernetes.io/projected/528b3ff9-2053-47e6-89f9-cc0b95774b4b-kube-api-access-746f2\") pod \"must-gather-wtt2q\" (UID: \"528b3ff9-2053-47e6-89f9-cc0b95774b4b\") " pod="openshift-must-gather-88m2b/must-gather-wtt2q"
Oct 30 06:41:37.471522 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.471487    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/528b3ff9-2053-47e6-89f9-cc0b95774b4b-must-gather-output\") pod \"must-gather-wtt2q\" (UID: \"528b3ff9-2053-47e6-89f9-cc0b95774b4b\") " pod="openshift-must-gather-88m2b/must-gather-wtt2q"
Oct 30 06:41:37.471754 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.471603    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-746f2\" (UniqueName: \"kubernetes.io/projected/528b3ff9-2053-47e6-89f9-cc0b95774b4b-kube-api-access-746f2\") pod \"must-gather-wtt2q\" (UID: \"528b3ff9-2053-47e6-89f9-cc0b95774b4b\") " pod="openshift-must-gather-88m2b/must-gather-wtt2q"
Oct 30 06:41:37.471907 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.471885    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/528b3ff9-2053-47e6-89f9-cc0b95774b4b-must-gather-output\") pod \"must-gather-wtt2q\" (UID: \"528b3ff9-2053-47e6-89f9-cc0b95774b4b\") " pod="openshift-must-gather-88m2b/must-gather-wtt2q"
Oct 30 06:41:37.493435 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.493401    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:41:37.497285 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.497266    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-746f2\" (UniqueName: \"kubernetes.io/projected/528b3ff9-2053-47e6-89f9-cc0b95774b4b-kube-api-access-746f2\") pod \"must-gather-wtt2q\" (UID: \"528b3ff9-2053-47e6-89f9-cc0b95774b4b\") " pod="openshift-must-gather-88m2b/must-gather-wtt2q"
Oct 30 06:41:37.604333 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.604254    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-88m2b/must-gather-wtt2q"
Oct 30 06:41:37.930331 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.930304    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-88m2b/must-gather-wtt2q"]
Oct 30 06:41:37.932783 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:41:37.932752    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod528b3ff9_2053_47e6_89f9_cc0b95774b4b.slice/crio-de8c7004ba2eb1a61c31dccb92490c8b93058a71e6cc6591aadd944db1c6a0ac WatchSource:0}: Error finding container de8c7004ba2eb1a61c31dccb92490c8b93058a71e6cc6591aadd944db1c6a0ac: Status 404 returned error can't find the container with id de8c7004ba2eb1a61c31dccb92490c8b93058a71e6cc6591aadd944db1c6a0ac
Oct 30 06:41:37.935079 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:37.935064    5066 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 06:41:38.392577 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:38.392486    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-88m2b/must-gather-wtt2q" event={"ID":"528b3ff9-2053-47e6-89f9-cc0b95774b4b","Type":"ContainerStarted","Data":"de8c7004ba2eb1a61c31dccb92490c8b93058a71e6cc6591aadd944db1c6a0ac"}
Oct 30 06:41:39.402514 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:39.402481    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-88m2b/must-gather-wtt2q" event={"ID":"528b3ff9-2053-47e6-89f9-cc0b95774b4b","Type":"ContainerStarted","Data":"ca5df11c588cc5da0f3b7610582f3aedd784e1bb85651e2fb19a92424313a9c3"}
Oct 30 06:41:39.402944 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:39.402522    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-88m2b/must-gather-wtt2q" event={"ID":"528b3ff9-2053-47e6-89f9-cc0b95774b4b","Type":"ContainerStarted","Data":"3ebd45a80bb56afedd83606367359856f3637aadef326c028cab9ecf02f37344"}
Oct 30 06:41:39.447397 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:39.447342    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-88m2b/must-gather-wtt2q" podStartSLOduration=1.661103856 podStartE2EDuration="2.447326512s" totalImagesPullingTime="786.222656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 06:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:41:39.445765497 +0000 UTC m=+4203.191050829" watchObservedRunningTime="2026-10-30 06:41:39.447326512 +0000 UTC m=+4203.192611863"
Oct 30 06:41:40.031121 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:40.031085    5066 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"]
Oct 30 06:41:40.043638 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:40.043606    5066 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-88c0edca-4svrf"]
Oct 30 06:41:40.400607 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:40.400513    5066 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f6acbe2-07b2-4529-9e84-238af4c53f19" path="/var/lib/kubelet/pods/7f6acbe2-07b2-4529-9e84-238af4c53f19/volumes"
Oct 30 06:41:42.011361 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:42.011323    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kkcc6_7b51a523-ec2a-45ec-b7a1-d0f1d0f778fa/global-pull-secret-syncer/0.log"
Oct 30 06:41:42.364028 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:42.363931    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-4sd8r_a0ea8679-d95b-49d6-a614-9bb4eb4ae029/connection-checker/0.log"
Oct 30 06:41:42.480449 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:42.480419    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-gh627_fcdb9d1a-474b-49f3-8683-f3874ad9d769/connection-checker/0.log"
Oct 30 06:41:42.609257 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:42.608219    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-k5d59_666a74c6-3979-480a-b4d7-4872c84b2413/connection-checker/0.log"
Oct 30 06:41:42.797195 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:42.797090    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log"
Oct 30 06:41:43.168490 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:43.168463    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-173.ec2.internal_ee22bb4b0b2b5043024e8d4645898710/haproxy/0.log"
Oct 30 06:41:44.260611 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:44.260216    5066 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bfpdf_7319282a-25bd-402e-a55c-8cc81da483c1/konnectivity-agent/0.log" limit=1024
Oct 30 06:41:50.279609 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:50.279392    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-4lplv_d25dce9a-0499-413d-99c7-0766aaf63fa1/manager/0.log"
Oct 30 06:41:50.390691 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:50.390658    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-6k4sq_3692525c-1db2-4a7a-ba04-c26bdcf52282/manager/0.log"
Oct 30 06:41:51.800153 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:51.799783    5066 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"]
Oct 30 06:41:51.804705 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:51.804679    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:51.837202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:51.837164    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"]
Oct 30 06:41:51.971144 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:51.971106    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e0150dfb-a998-4476-a203-24c96dbae6b3-podres\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:51.971324 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:51.971169    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0150dfb-a998-4476-a203-24c96dbae6b3-lib-modules\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:51.971324 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:51.971279    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e0150dfb-a998-4476-a203-24c96dbae6b3-proc\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:51.971417 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:51.971327    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q2h7\" (UniqueName: \"kubernetes.io/projected/e0150dfb-a998-4476-a203-24c96dbae6b3-kube-api-access-2q2h7\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:51.971417 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:51.971395    5066 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0150dfb-a998-4476-a203-24c96dbae6b3-sys\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:52.073202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.073104    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q2h7\" (UniqueName: \"kubernetes.io/projected/e0150dfb-a998-4476-a203-24c96dbae6b3-kube-api-access-2q2h7\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:52.073202 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.073191    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0150dfb-a998-4476-a203-24c96dbae6b3-sys\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:52.073467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.073272    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e0150dfb-a998-4476-a203-24c96dbae6b3-podres\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:52.073467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.073315    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0150dfb-a998-4476-a203-24c96dbae6b3-lib-modules\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:52.073467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.073318    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0150dfb-a998-4476-a203-24c96dbae6b3-sys\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:52.073467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.073386    5066 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e0150dfb-a998-4476-a203-24c96dbae6b3-proc\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:52.073467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.073406    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0150dfb-a998-4476-a203-24c96dbae6b3-lib-modules\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:52.073467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.073429    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e0150dfb-a998-4476-a203-24c96dbae6b3-podres\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:52.073832 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.073497    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e0150dfb-a998-4476-a203-24c96dbae6b3-proc\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:52.101697 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.101666    5066 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q2h7\" (UniqueName: \"kubernetes.io/projected/e0150dfb-a998-4476-a203-24c96dbae6b3-kube-api-access-2q2h7\") pod \"perf-node-gather-daemonset-z2lfz\" (UID: \"e0150dfb-a998-4476-a203-24c96dbae6b3\") " pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:52.124478 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.124443    5066 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:52.448132 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.448106    5066 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"]
Oct 30 06:41:52.449945 ip-10-0-136-173 kubenswrapper[5066]: W1030 06:41:52.449914    5066 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode0150dfb_a998_4476_a203_24c96dbae6b3.slice/crio-cf3b2b4c9e9d531ccf9a6febd3e27c3e733979303c06e07540a253244ea4bf7f WatchSource:0}: Error finding container cf3b2b4c9e9d531ccf9a6febd3e27c3e733979303c06e07540a253244ea4bf7f: Status 404 returned error can't find the container with id cf3b2b4c9e9d531ccf9a6febd3e27c3e733979303c06e07540a253244ea4bf7f
Oct 30 06:41:52.545094 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:52.545060    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz" event={"ID":"e0150dfb-a998-4476-a203-24c96dbae6b3","Type":"ContainerStarted","Data":"cf3b2b4c9e9d531ccf9a6febd3e27c3e733979303c06e07540a253244ea4bf7f"}
Oct 30 06:41:53.309972 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:53.309937    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-6xnts_bfa3e142-50e3-4b85-83fa-46f7dc8b7858/cluster-monitoring-operator/0.log"
Oct 30 06:41:53.553302 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:53.553266    5066 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz" event={"ID":"e0150dfb-a998-4476-a203-24c96dbae6b3","Type":"ContainerStarted","Data":"35ce0578d8e8741ac0259dcd0322009882bfff6a217746e571a727aaae516d8b"}
Oct 30 06:41:53.553489 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:53.553403    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:41:53.600194 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:53.600090    5066 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz" podStartSLOduration=2.6000702909999998 podStartE2EDuration="2.600070291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 06:41:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 06:41:53.598016896 +0000 UTC m=+4217.343302228" watchObservedRunningTime="2026-10-30 06:41:53.600070291 +0000 UTC m=+4217.345355625"
Oct 30 06:41:54.684677 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:54.684651    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zdfdw_5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4/node-exporter/0.log"
Oct 30 06:41:54.796507 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:54.796479    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zdfdw_5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4/kube-rbac-proxy/0.log"
Oct 30 06:41:54.908820 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:54.908794    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zdfdw_5447ce87-ff6b-4a21-bb0b-3bf21d26b0a4/init-textfile/0.log"
Oct 30 06:41:56.186975 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:56.186942    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-zzsn2_6f8f7445-3608-45fe-8964-81fc8fb06e56/prometheus-operator/0.log"
Oct 30 06:41:56.295316 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:56.295288    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-zzsn2_6f8f7445-3608-45fe-8964-81fc8fb06e56/kube-rbac-proxy/0.log"
Oct 30 06:41:56.412273 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:56.412243    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-vnqv7_6ab1819c-973d-4b13-a869-6a5f48af8a35/prometheus-operator-admission-webhook/0.log"
Oct 30 06:41:59.011656 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:59.011627    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-v6b7k_b0bdcd7b-f471-405a-87b2-ebedb8a1c950/networking-console-plugin/0.log"
Oct 30 06:41:59.575431 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:41:59.575400    5066 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-88m2b/perf-node-gather-daemonset-z2lfz"
Oct 30 06:42:00.087135 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:42:00.087082    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/1.log"
Oct 30 06:42:00.091255 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:42:00.091229    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-58dpm_ab824d2b-de9a-4384-a4a8-166f58808be8/console-operator/2.log"
Oct 30 06:42:01.124622 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:42:01.124562    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84ff984b45-g6jd8_945444af-d32f-4647-a40d-7d68802a9b9c/console/0.log"
Oct 30 06:42:01.259019 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:42:01.258988    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-8xfrp_40e94164-3716-4461-8d31-f93b3a27c0e1/download-server/0.log"
Oct 30 06:42:02.565439 ip-10-0-136-173 kubenswrapper[5066]: E1030 06:42:02.565391    5066 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6864g" containerName="extractor"
Oct 30 06:42:02.583818 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:42:02.583788    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-gshpq_2f92fa2c-9133-484f-b326-a12b2b618900/volume-data-source-validator/0.log"
Oct 30 06:42:04.997227 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:42:04.997200    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2_e1bf555f-5bd6-4b63-8c64-04d79c058cb4/istio-proxy/0.log"
Oct 30 06:42:05.252467 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:42:05.252402    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bzbfz_c46db00b-3c48-4d8d-844c-918d8126cc41/dns/0.log"
Oct 30 06:42:05.363793 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:42:05.363756    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bzbfz_c46db00b-3c48-4d8d-844c-918d8126cc41/kube-rbac-proxy/0.log"
Oct 30 06:42:05.832163 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:42:05.832131    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mtj5z_b4c2b487-ec63-4b07-a11c-781cb9afd37d/dns-node-resolver/0.log"
Oct 30 06:42:07.104277 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:42:07.104239    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6649cbddb-7n469_6e92e1cf-34ac-4b5b-a6c8-835ee435284c/registry/0.log"
Oct 30 06:42:07.455372 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:42:07.455323    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qw6mh_14a30f61-33a2-4f29-ba24-fc02c0c6cc15/node-ca/0.log"
Oct 30 06:42:09.365106 ip-10-0-136-173 kubenswrapper[5066]: I1030 06:42:09.365073    5066 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-59cf4dc6bfjxkq2_e1bf555f-5bd6-4b63-8c64-04d79c058cb4/istio-proxy/0.log"