Jul 23 00:23:16.068675 ip-10-0-133-215 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 00:23:16.540570 ip-10-0-133-215 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 00:23:16.540570 ip-10-0-133-215 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 00:23:16.540570 ip-10-0-133-215 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 00:23:16.540570 ip-10-0-133-215 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 00:23:16.540570 ip-10-0-133-215 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 00:23:16.541673 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.541553    2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 00:23:16.548741 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548718    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 00:23:16.548741 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548739    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 00:23:16.548741 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548744    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 00:23:16.548741 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548748    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548752    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548755    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548759    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548761    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548765    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548767    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548770    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548773    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548775    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548778    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548781    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548784    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548787    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548790    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548792    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548795    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548798    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548802    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 00:23:16.548903 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548805    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548809    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548812    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548815    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548818    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548821    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548824    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548827    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548830    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548833    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548841    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548845    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548848    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548851    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548853    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548857    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548860    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548862    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548866    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548868    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 00:23:16.549404 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548871    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548874    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548877    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548881    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548885    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548888    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548891    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548894    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548897    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548901    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548903    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548906    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548909    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548912    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548915    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548918    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548922    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548925    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548928    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 00:23:16.549965 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548931    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548934    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548936    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548939    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548941    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548944    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548947    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548950    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548954    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548957    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548960    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548963    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548965    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548968    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548971    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548973    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548976    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548979    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548982    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548985    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 00:23:16.550485 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548987    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548990    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548994    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548996    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.548999    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549003    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549006    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549009    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549011    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549014    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549017    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549020    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549022    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549025    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549028    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549031    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549034    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549036    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549849    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549858    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 00:23:16.550998 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549862    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549865    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549869    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549872    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549875    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549878    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549881    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549885    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549889    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549892    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549895    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549898    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549901    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549904    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549906    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549910    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549912    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549917    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549919    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549922    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 00:23:16.551532 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549925    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549928    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549931    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549933    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549936    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549939    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549941    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549944    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549949    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549952    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549956    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549959    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549963    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549966    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549970    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549973    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549976    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549979    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549982    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 00:23:16.552055 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549985    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549988    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549992    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.549996    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550000    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550003    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550006    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550009    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550012    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550015    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550019    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550022    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550025    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550027    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550030    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550033    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550036    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550038    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550041    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550044    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 00:23:16.552555 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550048    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550051    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550054    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550056    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550059    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550062    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550065    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550067    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550070    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550073    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550076    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550079    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550081    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550084    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550087    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550090    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550093    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550095    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550098    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550101    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 00:23:16.553078 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550104    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550106    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550110    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550113    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550115    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550118    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550121    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550124    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550126    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550129    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550131    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550134    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550137    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550139    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550142    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550145    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550148    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550151    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550570    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 00:23:16.553606 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550576    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550579    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550582    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550585    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550588    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550591    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550594    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550597    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550602    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550606    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550609    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550612    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550616    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550619    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550621    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550624    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550627    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550630    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550633    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 00:23:16.554100 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550636    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550638    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550641    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550644    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550647    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550650    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550652    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550655    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550657    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550660    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550663    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550666    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550669    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550672    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550675    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550677    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550680    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550683    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550686    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550689    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 00:23:16.554614 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550692    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550694    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550698    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550702    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550705    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550708    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550710    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550713    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550716    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550719    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550721    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550724    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550727    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550729    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550732    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550735    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550738    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550741    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550743    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550746    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 00:23:16.555126 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550749    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550752    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550755    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550758    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550760    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550763    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550766    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550769    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550772    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550774    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550777    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550780    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550783    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550785    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550788    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550792    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550798    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550801    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550804    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550807    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 00:23:16.555711 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550810    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550813    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550817    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550819    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550822    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550825    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550828    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550831    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550833    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550836    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550839    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550841    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550846    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550849    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550852    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550854    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550857    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550860    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.550863    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.550936    2561 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 00:23:16.556220 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.550943    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.550950    2561 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.550957    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.550963    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.550968    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.550975    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.550981    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.550985    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.550989    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.550993    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.550996    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551000    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551003    2561 flags.go:64] FLAG: --cgroup-root=""
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551007    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551010    2561 flags.go:64] FLAG: --client-ca-file=""
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551014    2561 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551017    2561 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551021    2561 flags.go:64] FLAG: --cluster-domain=""
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551025    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551028    2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551031    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551035    2561 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551039    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 00:23:16.556756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551042    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551047    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551051    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551054    2561 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551057    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551060    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551064    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551067    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551071    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551074    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551077    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551081    2561 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551084    2561 flags.go:64] FLAG: --enable-server="true"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551087    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551092    2561 flags.go:64] FLAG: --event-burst="100"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551096    2561 flags.go:64] FLAG: --event-qps="50"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551099    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551102    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551105    2561 flags.go:64] FLAG: --eviction-hard=""
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551109    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551112    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551116    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551119    2561 flags.go:64] FLAG: --eviction-soft=""
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551122    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551125    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 00:23:16.557375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551129    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551132    2561 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551135    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551144    2561 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551147    2561 flags.go:64] FLAG: --feature-gates=""
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551151    2561 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551154    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551158    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551162    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551166    2561 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551169    2561 flags.go:64] FLAG: --help="false"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551172    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-133-215.ec2.internal"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551175    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551178    2561 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551181    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551185    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551188    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551191    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551194    2561 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551197    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551200    2561 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551203    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551206    2561 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551219    2561 flags.go:64] FLAG: --kube-reserved=""
Jul 23 00:23:16.558053 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551222    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551226    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551229    2561 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551232    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551235    2561 flags.go:64] FLAG: --lock-file=""
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551238    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551241    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551244    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551250    2561 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551254    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551257    2561 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551262    2561 flags.go:64] FLAG: --logging-format="text"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551265    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551268    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551271    2561 flags.go:64] FLAG: --manifest-url=""
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551274    2561 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551280    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551283    2561 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551288    2561 flags.go:64] FLAG: --max-pods="110"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551291    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551294    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551297    2561 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551301    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551304    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551307    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 00:23:16.558709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551310    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551317    2561 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551320    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551324    2561 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551327    2561 flags.go:64] FLAG: --pod-cidr=""
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551330    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551336    2561 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551339    2561 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551342    2561 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551345    2561 flags.go:64] FLAG: --port="10250"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551348    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551351    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d23fdcb501052683"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551354    2561 flags.go:64] FLAG: --qos-reserved=""
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551357    2561 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551360    2561 flags.go:64] FLAG: --register-node="true"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551364    2561 flags.go:64] FLAG: --register-with-taints=""
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551367    2561 flags.go:64] FLAG: --registry-burst="10"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551374    2561 flags.go:64] FLAG: --registry-qps="5"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551378    2561 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551381    2561 flags.go:64] FLAG: --reserved-memory=""
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551385    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551388    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551392    2561 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551394    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551399    2561 flags.go:64] FLAG: --runonce="false"
Jul 23 00:23:16.559365 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551402    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551405    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551408    2561 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551411    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551414    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551417    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551420    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551423    2561 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551426    2561 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551429    2561 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551433    2561 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551451    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551454    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551458    2561 flags.go:64] FLAG: --system-cgroups=""
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551461    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551466    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551469    2561 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551472    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551477    2561 flags.go:64] FLAG: --tls-min-version=""
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551480    2561 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551482    2561 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551485    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551488    2561 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551491    2561 flags.go:64] FLAG: --v="2"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551496    2561 flags.go:64] FLAG: --version="false"
Jul 23 00:23:16.560049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551508    2561 flags.go:64] FLAG: --vmodule=""
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551513    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.551517    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551652    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551656    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551660    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551664    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551667    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551670    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551673    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551675    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551678    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551681    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551684    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551687    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551689    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551692    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551695    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551698    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551701    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 00:23:16.560708 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551703    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551706    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551709    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551712    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551714    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551717    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551720    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551722    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551725    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551728    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551731    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551733    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551737    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551741    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551743    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551746    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551749    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551752    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551757    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 00:23:16.561386 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551760    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551762    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551765    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551768    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551771    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551773    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551776    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551779    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551781    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551784    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551787    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551789    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551792    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551795    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551797    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551800    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551803    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551806    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551808    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551811    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 00:23:16.561971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551813    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551816    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551819    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551821    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551824    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551828    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551831    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551835    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551838    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551840    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551843    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551847    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551850    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551854    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551857    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551859    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551864    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551868    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551871    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551874    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 00:23:16.562504 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551877    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551880    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551883    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551885    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551888    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551891    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551894    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551897    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551899    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551902    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551906    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551910    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551913    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551917    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551921    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551924    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551927    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551931    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551934    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 00:23:16.563036 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551937    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551941    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551944    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.551946    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.552935    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.560272    2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.560294    2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.560322    2561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.560328    2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560392    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560398    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560402    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560405    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560409    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560412    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 00:23:16.563538 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560415    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560418    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560421    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560424    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560427    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560430    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560432    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560435    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560461    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560464    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560467    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560469    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560474    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560478    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560481    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560484    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560487    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560489    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560492    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560495    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 00:23:16.563940 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560498    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560501    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560504    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560514    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560517    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560520    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560523    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560526    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560529    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560532    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560534    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560537    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560540    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560543    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560546    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560549    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560551    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560554    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560557    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560559    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 00:23:16.564476 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560562    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560565    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560568    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560572    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560576    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560579    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560582    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560584    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560587    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560590    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560592    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560595    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560598    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560600    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560603    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560607    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560616    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560619    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560621    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560624    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 00:23:16.564995 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560627    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560630    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560632    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560635    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560638    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560642    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560646    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560649    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560652    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560655    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560658    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560660    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560663    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560666    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560669    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560671    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560674    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560677    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560679    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 00:23:16.565535 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560682    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560685    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560688    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560690    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560693    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560696    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560698    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560701    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560704    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560708    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560717    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560720    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560723    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560726    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.560731    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 00:23:16.566029 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560884    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560890    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560893    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560896    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560900    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560903    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560906    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560910    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560913    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560916    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560919    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560921    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560924    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560926    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560929    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560932    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560935    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560937    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560940    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560943    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 00:23:16.566420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560946    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560949    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560952    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560954    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560958    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560960    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560964    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560967    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560976    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560979    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560981    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560984    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560987    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560989    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560992    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560994    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.560997    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561000    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561003    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561006    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 00:23:16.566951 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561009    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561011    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561014    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561017    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561020    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561023    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561026    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561029    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561032    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561034    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561037    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561040    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561044    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561048    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561051    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561055    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561058    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561061    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561065    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 00:23:16.567481 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561069    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561072    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561081    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561084    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561087    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561089    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561092    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561095    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561098    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561100    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561103    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561106    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561109    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561112    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561114    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561117    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561120    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561122    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561125    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561127    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 00:23:16.567971 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561130    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561133    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561135    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561138    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561140    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561143    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561146    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561149    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561151    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561154    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561156    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561159    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561162    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561165    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561167    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561176    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561179    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561182    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561186    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 00:23:16.568499 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:16.561189    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 00:23:16.569016 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.561194    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 00:23:16.569016 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.561318    2561 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 00:23:16.569016 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.563596    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 00:23:16.569016 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.565537    2561 server.go:1019] "Starting client certificate rotation"
Jul 23 00:23:16.569016 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.565644    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 00:23:16.569016 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.565676    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 00:23:16.569190 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.569094    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 00:23:16.572252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.572222    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 00:23:16.585512 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.585494    2561 log.go:25] "Validated CRI v1 runtime API"
Jul 23 00:23:16.587938 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.587921    2561 log.go:25] "Validated CRI v1 image API"
Jul 23 00:23:16.588844 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.588828    2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 00:23:16.592258 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.592226    2561 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8898fffe-4d1c-4db6-b12b-990e40c563b0:/dev/nvme0n1p4 f746c290-8f8f-4462-953a-b4c86864eb31:/dev/nvme0n1p3]
Jul 23 00:23:16.592336 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.592255    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 00:23:16.597946 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.597830    2561 manager.go:216] Machine: {Timestamp:2026-07-23 00:23:16.596255118 +0000 UTC m=+0.402666344 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3114635 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2120555f9b747cf961ca4dacdd1944 SystemUUID:ec212055-5f9b-747c-f961-ca4dacdd1944 BootID:391954ba-eaf3-4f11-afcd-9ea28bd40eaf Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:80:db:01:ef:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:80:db:01:ef:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:22:a3:a9:eb:f2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 00:23:16.597946 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.597943    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 00:23:16.598076 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.598025    2561 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 00:23:16.601011 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.600965    2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 00:23:16.601209 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.601018    2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-215.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 00:23:16.601748 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.601738    2561 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 00:23:16.601783 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.601751    2561 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 00:23:16.601783 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.601759    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 00:23:16.602420 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.602407    2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 00:23:16.602539 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.602524    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 00:23:16.603741 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.603728    2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 00:23:16.606645 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.606633    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 00:23:16.606770 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.606760    2561 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 00:23:16.606805 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.606794    2561 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 00:23:16.606838 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.606808    2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 00:23:16.606838 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.606834    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 00:23:16.606896 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.606852    2561 kubelet.go:398] "Adding apiserver pod source"
Jul 23 00:23:16.606896 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.606865    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 00:23:16.607362 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.607348    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 00:23:16.607408 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.607378    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 00:23:16.610895 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.610879    2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 00:23:16.613359 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.613345    2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 00:23:16.613414 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.613378    2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 00:23:16.614961 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.614948    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 00:23:16.615006 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.614968    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 00:23:16.615006 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.614975    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 00:23:16.615006 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.614981    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 00:23:16.615006 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.614990    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 00:23:16.615006 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.614996    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 00:23:16.615006 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.615005    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 00:23:16.615170 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.615011    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 00:23:16.615170 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.615019    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 00:23:16.615170 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.615024    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 00:23:16.615170 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.615034    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 00:23:16.615170 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.615040    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 00:23:16.615654 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.615639    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 00:23:16.615654 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.615651    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 00:23:16.618017 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.617993    2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-215.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 00:23:16.618623 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.618603    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 00:23:16.618710 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.618608    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 00:23:16.619017 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.619004    2561 server.go:1268] "Started kubelet"
Jul 23 00:23:16.623296 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.623175    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 00:23:16.623778 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.623718    2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 00:23:16.623868 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.623802    2561 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 00:23:16.624313 ip-10-0-133-215 systemd[1]: Started Kubernetes Kubelet.
Jul 23 00:23:16.625187 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.625170    2561 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 00:23:16.627482 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.627457    2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 00:23:16.630475 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.630454    2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 00:23:16.635125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.635105    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 00:23:16.635713 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.635548    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 00:23:16.635713 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.635661    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:16.636178 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.636159    2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 00:23:16.636178 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.636165    2561 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 00:23:16.636334 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.636200    2561 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 00:23:16.636334 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.636299    2561 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 00:23:16.636334 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.636309    2561 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 00:23:16.636463 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.636402    2561 factory.go:55] Registering systemd factory
Jul 23 00:23:16.636463 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.636419    2561 factory.go:223] Registration of the systemd container factory successfully
Jul 23 00:23:16.636670 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.636656    2561 factory.go:153] Registering CRI-O factory
Jul 23 00:23:16.636705 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.636672    2561 factory.go:223] Registration of the crio container factory successfully
Jul 23 00:23:16.636735 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.636716    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 00:23:16.636780 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.636737    2561 factory.go:103] Registering Raw factory
Jul 23 00:23:16.636780 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.636751    2561 manager.go:1196] Started watching for new ooms in manager
Jul 23 00:23:16.637119 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.637108    2561 manager.go:349] Starting recovery of all containers
Jul 23 00:23:16.640091 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.640061    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 00:23:16.641228 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.640118    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-215.ec2.internal.18c4c36654993ff5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-215.ec2.internal,UID:ip-10-0-133-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-215.ec2.internal,},FirstTimestamp:2026-07-23 00:23:16.618969077 +0000 UTC m=+0.425380306,LastTimestamp:2026-07-23 00:23:16.618969077 +0000 UTC m=+0.425380306,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-215.ec2.internal,}"
Jul 23 00:23:16.641352 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.641325    2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 00:23:16.649242 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.649224    2561 manager.go:354] Recovery completed
Jul 23 00:23:16.653738 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.653726    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:23:16.655533 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.655502    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:23:16.655603 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.655549    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:23:16.655603 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.655564    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:23:16.656173 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.656159    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 00:23:16.656173 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.656172    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 00:23:16.656289 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.656190    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 00:23:16.656289 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.656194    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:23:16.657275 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.657259    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:23:16.657322 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.657291    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:23:16.657322 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.657308    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:23:16.657605 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.657593    2561 policy_none.go:49] "None policy: Start"
Jul 23 00:23:16.657650 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.657614    2561 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 00:23:16.657650 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.657625    2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 00:23:16.658472 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.658355    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-215.ec2.internal.18c4c36656c724ca  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-215.ec2.internal,UID:ip-10-0-133-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-215.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-215.ec2.internal,},FirstTimestamp:2026-07-23 00:23:16.65553121 +0000 UTC m=+0.461942444,LastTimestamp:2026-07-23 00:23:16.65553121 +0000 UTC m=+0.461942444,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-215.ec2.internal,}"
Jul 23 00:23:16.658813 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.658799    2561 policy_none.go:47] "Start"
Jul 23 00:23:16.666209 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.666187    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zqjhq"
Jul 23 00:23:16.666411 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.666332    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-215.ec2.internal.18c4c36656c7898a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-215.ec2.internal,UID:ip-10-0-133-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-215.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-215.ec2.internal,},FirstTimestamp:2026-07-23 00:23:16.655557002 +0000 UTC m=+0.461968238,LastTimestamp:2026-07-23 00:23:16.655557002 +0000 UTC m=+0.461968238,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-215.ec2.internal,}"
Jul 23 00:23:16.674845 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.674728    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zqjhq"
Jul 23 00:23:16.679131 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.679064    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-215.ec2.internal.18c4c36656c7b7b9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-215.ec2.internal,UID:ip-10-0-133-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-215.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-215.ec2.internal,},FirstTimestamp:2026-07-23 00:23:16.655568825 +0000 UTC m=+0.461980052,LastTimestamp:2026-07-23 00:23:16.655568825 +0000 UTC m=+0.461980052,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-215.ec2.internal,}"
Jul 23 00:23:16.700077 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.700055    2561 manager.go:337] "Starting Device Plugin manager"
Jul 23 00:23:16.700187 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.700101    2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 00:23:16.700187 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.700113    2561 server.go:87] "Starting device plugin registration server"
Jul 23 00:23:16.700402 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.700377    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 00:23:16.700452 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.700392    2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 00:23:16.700537 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.700512    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 00:23:16.700645 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.700607    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 00:23:16.700645 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.700615    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 00:23:16.701244 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.701224    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 00:23:16.701325 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.701275    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:16.779691 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.779651    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 00:23:16.781013 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.780974    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 00:23:16.781013 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.781013    2561 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 00:23:16.781152 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.781037    2561 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 00:23:16.781152 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.781077    2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 00:23:16.783699 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.783677    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 00:23:16.801472 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.801403    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:23:16.802296 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.802280    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:23:16.802349 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.802312    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:23:16.802349 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.802326    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:23:16.802432 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.802350    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-215.ec2.internal"
Jul 23 00:23:16.811808 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.811793    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-215.ec2.internal"
Jul 23 00:23:16.811890 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.811814    2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-215.ec2.internal\": node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:16.838194 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.838173    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:16.882531 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.882487    2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"]
Jul 23 00:23:16.882683 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.882570    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:23:16.883721 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.883704    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:23:16.883799 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.883735    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:23:16.883799 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.883745    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:23:16.885196 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.885182    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:23:16.885300 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.885284    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:16.885340 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.885317    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:23:16.885995 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.885969    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:23:16.885995 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.885982    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:23:16.885995 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.885997    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:23:16.886126 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.886006    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:23:16.886126 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.886009    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:23:16.886126 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.886020    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:23:16.887300 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.887282    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:16.887368 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.887310    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:23:16.888094 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.888075    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:23:16.888160 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.888102    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:23:16.888160 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.888118    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:23:16.913199 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.913174    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-215.ec2.internal\" not found" node="ip-10-0-133-215.ec2.internal"
Jul 23 00:23:16.917843 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.917825    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-215.ec2.internal\" not found" node="ip-10-0-133-215.ec2.internal"
Jul 23 00:23:16.937586 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.937563    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c98ef64082e55147c9ba40e982d64173-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal\" (UID: \"c98ef64082e55147c9ba40e982d64173\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:16.937659 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.937588    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dcbe9759263d750ea2db6eb7b4e9f691-config\") pod \"kube-apiserver-proxy-ip-10-0-133-215.ec2.internal\" (UID: \"dcbe9759263d750ea2db6eb7b4e9f691\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:16.937659 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:16.937606    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c98ef64082e55147c9ba40e982d64173-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal\" (UID: \"c98ef64082e55147c9ba40e982d64173\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:16.938684 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:16.938669    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:17.038586 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.038556    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c98ef64082e55147c9ba40e982d64173-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal\" (UID: \"c98ef64082e55147c9ba40e982d64173\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:17.038710 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.038589    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dcbe9759263d750ea2db6eb7b4e9f691-config\") pod \"kube-apiserver-proxy-ip-10-0-133-215.ec2.internal\" (UID: \"dcbe9759263d750ea2db6eb7b4e9f691\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:17.038710 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.038619    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c98ef64082e55147c9ba40e982d64173-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal\" (UID: \"c98ef64082e55147c9ba40e982d64173\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:17.038710 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.038664    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c98ef64082e55147c9ba40e982d64173-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal\" (UID: \"c98ef64082e55147c9ba40e982d64173\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:17.038710 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.038679    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dcbe9759263d750ea2db6eb7b4e9f691-config\") pod \"kube-apiserver-proxy-ip-10-0-133-215.ec2.internal\" (UID: \"dcbe9759263d750ea2db6eb7b4e9f691\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:17.038855 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:17.038707    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:17.038855 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.038679    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c98ef64082e55147c9ba40e982d64173-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal\" (UID: \"c98ef64082e55147c9ba40e982d64173\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:17.139207 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:17.139133    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:17.218071 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.218033    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:17.220788 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.220767    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:17.239586 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:17.239560    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:17.342502 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:17.342453    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:17.443126 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:17.443043    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:17.543445 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:17.543418    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:17.557130 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.557108    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 00:23:17.564196 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.564179    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 00:23:17.564334 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.564312    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 00:23:17.564371 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.564349    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 00:23:17.644252 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:17.644222    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:17.678718 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.678680    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 00:18:16 +0000 UTC" deadline="2028-05-01 16:28:43.489766238 +0000 UTC"
Jul 23 00:23:17.678718 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.678715    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15568h5m25.811054489s"
Jul 23 00:23:17.744579 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:17.744551    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:17.794268 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:17.794230    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcbe9759263d750ea2db6eb7b4e9f691.slice/crio-409e9550a9173d203d24a174f3c4dc7452c503de1d85c6a835cf1ca154b9fcac WatchSource:0}: Error finding container 409e9550a9173d203d24a174f3c4dc7452c503de1d85c6a835cf1ca154b9fcac: Status 404 returned error can't find the container with id 409e9550a9173d203d24a174f3c4dc7452c503de1d85c6a835cf1ca154b9fcac
Jul 23 00:23:17.794686 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:17.794659    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98ef64082e55147c9ba40e982d64173.slice/crio-dc2b2d69bcaf2fbea0e69f52bd9523e19f11c11f9080338552a2402feae7e932 WatchSource:0}: Error finding container dc2b2d69bcaf2fbea0e69f52bd9523e19f11c11f9080338552a2402feae7e932: Status 404 returned error can't find the container with id dc2b2d69bcaf2fbea0e69f52bd9523e19f11c11f9080338552a2402feae7e932
Jul 23 00:23:17.798776 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.798754    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 00:23:17.803612 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.803595    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:23:17.804626 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.804608    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:23:17.804695 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.804640    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:23:17.804695 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:17.804667    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:23:17.845009 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:17.844974    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:17.945954 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:17.945885    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-215.ec2.internal\" not found"
Jul 23 00:23:18.004259 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.004229    2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 00:23:18.036847 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.036813    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:18.045475 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.045452    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 00:23:18.047461 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.047437    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal"
Jul 23 00:23:18.059514 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.059502    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 00:23:18.079215 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.079189    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 00:23:18.609174 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.609148    2561 apiserver.go:52] "Watching apiserver"
Jul 23 00:23:18.618637 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.618609    2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 00:23:18.621106 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.621072    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-drxfq","openshift-image-registry/node-ca-hsf6v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal","openshift-network-diagnostics/network-check-target-nzvr2","openshift-network-operator/iptables-alerter-jksgp","kube-system/global-pull-secret-syncer-l8zzv","kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal","openshift-dns/node-resolver-5z66q","openshift-multus/multus-additional-cni-plugins-b76l8","openshift-multus/multus-zgm69","openshift-multus/network-metrics-daemon-6dn2c","openshift-ovn-kubernetes/ovnkube-node-g9g57","kube-system/konnectivity-agent-vgdrk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"]
Jul 23 00:23:18.622686 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.622665    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jksgp"
Jul 23 00:23:18.624280 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.624127    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hsf6v"
Jul 23 00:23:18.625301 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.625239    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.625809 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.625658    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-54zb7\""
Jul 23 00:23:18.625809 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.625747    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 00:23:18.625976 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.625809    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 00:23:18.625976 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.625968    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 00:23:18.626671 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.626653    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:18.626768 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:18.626739    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:18.626887 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.626862    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 00:23:18.626887 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.626881    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 00:23:18.626976 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.626961    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 00:23:18.627274 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.627213    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2kxk6\""
Jul 23 00:23:18.628485 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.627999    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 00:23:18.628485 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.628121    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cghjk\""
Jul 23 00:23:18.628485 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.628349    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 00:23:18.628678 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.628536    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 00:23:18.628678 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.628646    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 00:23:18.628915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.628897    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:18.629037 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:18.629002    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:18.630327 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.629853    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5z66q"
Jul 23 00:23:18.631168 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.631151    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.632293 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.632272    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 00:23:18.632603 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.632582    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.632697 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.632626    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fwwbr\""
Jul 23 00:23:18.632995 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.632912    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 00:23:18.633885 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.633864    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:18.633992 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:18.633947    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:18.634299 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.634072    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cgck4\""
Jul 23 00:23:18.634299 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.634081    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 00:23:18.634299 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.634143    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 00:23:18.635849 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.635829    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 00:23:18.636006 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.635986    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 00:23:18.636465 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.636149    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 00:23:18.636465 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.636207    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.638409 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.637407    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vgdrk"
Jul 23 00:23:18.639877 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.638589    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6zsrf\""
Jul 23 00:23:18.640661 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.639979    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 00:23:18.640661 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.640094    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 00:23:18.640661 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.640176    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 00:23:18.640841 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.640707    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-29frj\""
Jul 23 00:23:18.640841 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.640761    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 00:23:18.642121 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.641668    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.642121 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.641745    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 00:23:18.642121 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.641963    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 00:23:18.642315 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.642303    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 00:23:18.642626 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.642565    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fxg4c\""
Jul 23 00:23:18.642723 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.642657    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 00:23:18.644846 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.644711    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 00:23:18.644846 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.644734    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 00:23:18.645009 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.644980    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dcx9m\""
Jul 23 00:23:18.645249 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.645229    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 00:23:18.649861 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.649838    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-cni-netd\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.649939 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.649876    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-var-lib-kubelet\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.649939 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.649906    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f45wr\" (UniqueName: \"kubernetes.io/projected/db83e8c2-912c-42dc-8397-b428ebca65b8-kube-api-access-f45wr\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.649939 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.649932    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/64edf751-2ca9-432a-a626-2c23a5f79032-kubelet-config\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:18.650051 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.649958    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-sysconfig\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.650051 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.649993    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-system-cni-dir\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.650051 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650023    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-multus-cni-dir\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.650051 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650042    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-etc-kubernetes\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.650175 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650057    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-kubernetes\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.650175 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650073    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-cni-bin\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.650175 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650089    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db83e8c2-912c-42dc-8397-b428ebca65b8-multus-daemon-config\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.650175 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650104    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/199bdf35-3af4-4b29-8e0d-1c68fc3ec572-host-slash\") pod \"iptables-alerter-jksgp\" (UID: \"199bdf35-3af4-4b29-8e0d-1c68fc3ec572\") " pod="openshift-network-operator/iptables-alerter-jksgp"
Jul 23 00:23:18.650175 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650119    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/410e2099-4588-45d5-a93c-cb4066032492-serviceca\") pod \"node-ca-hsf6v\" (UID: \"410e2099-4588-45d5-a93c-cb4066032492\") " pod="openshift-image-registry/node-ca-hsf6v"
Jul 23 00:23:18.650175 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650133    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd2bec1a-032b-42cd-9555-7154268c805c-system-cni-dir\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.650175 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650152    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-systemd\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.650372 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650175    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-run-netns\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.650372 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650192    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-run-ovn-kubernetes\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.650372 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650207    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.650372 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650252    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:18.650372 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650281    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:18.650372 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650297    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd2bec1a-032b-42cd-9555-7154268c805c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.650372 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650333    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cd2bec1a-032b-42cd-9555-7154268c805c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.650625 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650384    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-env-overrides\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.650625 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650421    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-os-release\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.650625 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650460    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-run-k8s-cni-cncf-io\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.650625 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650484    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd2bec1a-032b-42cd-9555-7154268c805c-cnibin\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.650625 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650500    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-etc-openvswitch\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.650625 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650517    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/199bdf35-3af4-4b29-8e0d-1c68fc3ec572-iptables-alerter-script\") pod \"iptables-alerter-jksgp\" (UID: \"199bdf35-3af4-4b29-8e0d-1c68fc3ec572\") " pod="openshift-network-operator/iptables-alerter-jksgp"
Jul 23 00:23:18.650625 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650544    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/410e2099-4588-45d5-a93c-cb4066032492-host\") pod \"node-ca-hsf6v\" (UID: \"410e2099-4588-45d5-a93c-cb4066032492\") " pod="openshift-image-registry/node-ca-hsf6v"
Jul 23 00:23:18.650625 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650566    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd2bec1a-032b-42cd-9555-7154268c805c-os-release\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.650625 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650585    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-run\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.650625 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650600    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-sys\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.650625 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650621    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zdgs\" (UniqueName: \"kubernetes.io/projected/7c65b730-bf89-4090-b8a7-cad9a31bf810-kube-api-access-7zdgs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650645    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-run-systemd\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650661    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6d6l\" (UniqueName: \"kubernetes.io/projected/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-kube-api-access-g6d6l\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650684    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-multus-socket-dir-parent\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650716    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-multus-conf-dir\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650733    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-run-multus-certs\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650750    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-sysctl-d\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650772    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-slash\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650798    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-cnibin\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650820    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b4e497b3-17fb-4e64-bd26-b0aec2019720-agent-certs\") pod \"konnectivity-agent-vgdrk\" (UID: \"b4e497b3-17fb-4e64-bd26-b0aec2019720\") " pod="kube-system/konnectivity-agent-vgdrk"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650838    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gz67\" (UniqueName: \"kubernetes.io/projected/199bdf35-3af4-4b29-8e0d-1c68fc3ec572-kube-api-access-8gz67\") pod \"iptables-alerter-jksgp\" (UID: \"199bdf35-3af4-4b29-8e0d-1c68fc3ec572\") " pod="openshift-network-operator/iptables-alerter-jksgp"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650859    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4fdfd2d4-f612-4ee0-b4cd-66417c138705-hosts-file\") pod \"node-resolver-5z66q\" (UID: \"4fdfd2d4-f612-4ee0-b4cd-66417c138705\") " pod="openshift-dns/node-resolver-5z66q"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650874    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd2bec1a-032b-42cd-9555-7154268c805c-cni-binary-copy\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650889    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cd2bec1a-032b-42cd-9555-7154268c805c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650905    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-lib-modules\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650926    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-tuned\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.651117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.650975    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-ovn-node-metrics-cert\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651012    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-var-lib-cni-bin\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651038    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb7n2\" (UniqueName: \"kubernetes.io/projected/4fdfd2d4-f612-4ee0-b4cd-66417c138705-kube-api-access-cb7n2\") pod \"node-resolver-5z66q\" (UID: \"4fdfd2d4-f612-4ee0-b4cd-66417c138705\") " pod="openshift-dns/node-resolver-5z66q"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651063    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-sysctl-conf\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651105    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-host\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651126    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-run-openvswitch\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651158    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-log-socket\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651211    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-ovnkube-config\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651247    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-modprobe-d\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651273    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-var-lib-kubelet\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651298    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzhdq\" (UniqueName: \"kubernetes.io/projected/cd2bec1a-032b-42cd-9555-7154268c805c-kube-api-access-bzhdq\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651323    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-node-log\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651347    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db83e8c2-912c-42dc-8397-b428ebca65b8-cni-binary-copy\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651409    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-var-lib-cni-multus\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651443    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-hostroot\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651493    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/64edf751-2ca9-432a-a626-2c23a5f79032-dbus\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651526    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fdfd2d4-f612-4ee0-b4cd-66417c138705-tmp-dir\") pod \"node-resolver-5z66q\" (UID: \"4fdfd2d4-f612-4ee0-b4cd-66417c138705\") " pod="openshift-dns/node-resolver-5z66q"
Jul 23 00:23:18.651880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651551    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmzxj\" (UniqueName: \"kubernetes.io/projected/7be69b0c-5f23-4e59-a484-022efa234e0e-kube-api-access-mmzxj\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.652701 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651575    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-ovnkube-script-lib\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.652701 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651600    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9646\" (UniqueName: \"kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646\") pod \"network-check-target-nzvr2\" (UID: \"c5eeb225-2f11-4dd2-b57f-933946195251\") " pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:18.652701 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651622    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-run-ovn\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.652701 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651647    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7be69b0c-5f23-4e59-a484-022efa234e0e-tmp\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.652701 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651669    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-kubelet\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.652701 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651693    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-systemd-units\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.652701 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651716    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-run-netns\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.652701 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651739    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b4e497b3-17fb-4e64-bd26-b0aec2019720-konnectivity-ca\") pod \"konnectivity-agent-vgdrk\" (UID: \"b4e497b3-17fb-4e64-bd26-b0aec2019720\") " pod="kube-system/konnectivity-agent-vgdrk"
Jul 23 00:23:18.652701 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651806    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf7jg\" (UniqueName: \"kubernetes.io/projected/410e2099-4588-45d5-a93c-cb4066032492-kube-api-access-kf7jg\") pod \"node-ca-hsf6v\" (UID: \"410e2099-4588-45d5-a93c-cb4066032492\") " pod="openshift-image-registry/node-ca-hsf6v"
Jul 23 00:23:18.652701 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651835    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-var-lib-openvswitch\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.652701 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.651932    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 00:23:18.657325 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.657305    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 00:23:18.670617 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.670596    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pg284"
Jul 23 00:23:18.678487 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.678465    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pg284"
Jul 23 00:23:18.737242 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.737210    2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 00:23:18.752537 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752499    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-tuned\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.752729 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752551    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-ovn-node-metrics-cert\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.752729 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752588    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-var-lib-cni-bin\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.752729 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752617    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb7n2\" (UniqueName: \"kubernetes.io/projected/4fdfd2d4-f612-4ee0-b4cd-66417c138705-kube-api-access-cb7n2\") pod \"node-resolver-5z66q\" (UID: \"4fdfd2d4-f612-4ee0-b4cd-66417c138705\") " pod="openshift-dns/node-resolver-5z66q"
Jul 23 00:23:18.752729 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752642    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.752729 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752665    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-socket-dir\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.752729 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752695    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-sysctl-conf\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.752729 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752717    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-host\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752730    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-var-lib-cni-bin\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752741    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-run-openvswitch\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752779    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-log-socket\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752778    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-run-openvswitch\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752807    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-ovnkube-config\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752841    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-log-socket\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752875    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-etc-selinux\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752891    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-host\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752920    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-modprobe-d\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752924    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752991    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-var-lib-kubelet\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.752947    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-var-lib-kubelet\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753063    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzhdq\" (UniqueName: \"kubernetes.io/projected/cd2bec1a-032b-42cd-9555-7154268c805c-kube-api-access-bzhdq\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753096    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-node-log\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753106    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-modprobe-d\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.753125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753121    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db83e8c2-912c-42dc-8397-b428ebca65b8-cni-binary-copy\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753140    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-sysctl-conf\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753146    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-var-lib-cni-multus\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753178    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-hostroot\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753185    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-var-lib-cni-multus\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753201    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/64edf751-2ca9-432a-a626-2c23a5f79032-dbus\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753232    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fdfd2d4-f612-4ee0-b4cd-66417c138705-tmp-dir\") pod \"node-resolver-5z66q\" (UID: \"4fdfd2d4-f612-4ee0-b4cd-66417c138705\") " pod="openshift-dns/node-resolver-5z66q"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753239    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-node-log\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753258    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmzxj\" (UniqueName: \"kubernetes.io/projected/7be69b0c-5f23-4e59-a484-022efa234e0e-kube-api-access-mmzxj\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753308    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-ovnkube-script-lib\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753332    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9646\" (UniqueName: \"kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646\") pod \"network-check-target-nzvr2\" (UID: \"c5eeb225-2f11-4dd2-b57f-933946195251\") " pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753358    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-device-dir\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753400    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-run-ovn\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.753815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.753422    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7be69b0c-5f23-4e59-a484-022efa234e0e-tmp\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755000    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db83e8c2-912c-42dc-8397-b428ebca65b8-cni-binary-copy\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755191    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-kubelet\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755241    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-systemd-units\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755280    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-run-netns\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755307    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b4e497b3-17fb-4e64-bd26-b0aec2019720-konnectivity-ca\") pod \"konnectivity-agent-vgdrk\" (UID: \"b4e497b3-17fb-4e64-bd26-b0aec2019720\") " pod="kube-system/konnectivity-agent-vgdrk"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755343    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf7jg\" (UniqueName: \"kubernetes.io/projected/410e2099-4588-45d5-a93c-cb4066032492-kube-api-access-kf7jg\") pod \"node-ca-hsf6v\" (UID: \"410e2099-4588-45d5-a93c-cb4066032492\") " pod="openshift-image-registry/node-ca-hsf6v"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755369    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-ovnkube-config\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755397    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-var-lib-openvswitch\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755405    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-run-ovn\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755434    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-cni-netd\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755490    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-var-lib-kubelet\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755488    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-cni-netd\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755528    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f45wr\" (UniqueName: \"kubernetes.io/projected/db83e8c2-912c-42dc-8397-b428ebca65b8-kube-api-access-f45wr\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755572    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/64edf751-2ca9-432a-a626-2c23a5f79032-kubelet-config\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755571    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/64edf751-2ca9-432a-a626-2c23a5f79032-dbus\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755611    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-sysconfig\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755667    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-system-cni-dir\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.755915 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755769    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-multus-cni-dir\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755801    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-etc-kubernetes\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755837    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-kubernetes\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755870    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-kubelet\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755984    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-cni-bin\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755872    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-cni-bin\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.755193    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-hostroot\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.756049    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-run-netns\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.756088    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-sysconfig\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.756004    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-systemd-units\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.756151    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/64edf751-2ca9-432a-a626-2c23a5f79032-kubelet-config\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.756258    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-multus-cni-dir\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.756323    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-system-cni-dir\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.756526    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-var-lib-kubelet\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.756750    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-var-lib-openvswitch\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.756882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.756870    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-kubernetes\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.756887    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db83e8c2-912c-42dc-8397-b428ebca65b8-multus-daemon-config\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.756922    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-etc-kubernetes\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.756088    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db83e8c2-912c-42dc-8397-b428ebca65b8-multus-daemon-config\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757077    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/199bdf35-3af4-4b29-8e0d-1c68fc3ec572-host-slash\") pod \"iptables-alerter-jksgp\" (UID: \"199bdf35-3af4-4b29-8e0d-1c68fc3ec572\") " pod="openshift-network-operator/iptables-alerter-jksgp"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757108    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/410e2099-4588-45d5-a93c-cb4066032492-serviceca\") pod \"node-ca-hsf6v\" (UID: \"410e2099-4588-45d5-a93c-cb4066032492\") " pod="openshift-image-registry/node-ca-hsf6v"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757135    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd2bec1a-032b-42cd-9555-7154268c805c-system-cni-dir\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757167    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-systemd\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757161    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fdfd2d4-f612-4ee0-b4cd-66417c138705-tmp-dir\") pod \"node-resolver-5z66q\" (UID: \"4fdfd2d4-f612-4ee0-b4cd-66417c138705\") " pod="openshift-dns/node-resolver-5z66q"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757198    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-run-netns\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757243    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-run-netns\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757248    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-run-ovn-kubernetes\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757287    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/199bdf35-3af4-4b29-8e0d-1c68fc3ec572-host-slash\") pod \"iptables-alerter-jksgp\" (UID: \"199bdf35-3af4-4b29-8e0d-1c68fc3ec572\") " pod="openshift-network-operator/iptables-alerter-jksgp"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757333    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-run-ovn-kubernetes\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757288    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757386    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757399    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-systemd\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.757608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757422    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-registration-dir\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757462    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757467    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b4e497b3-17fb-4e64-bd26-b0aec2019720-konnectivity-ca\") pod \"konnectivity-agent-vgdrk\" (UID: \"b4e497b3-17fb-4e64-bd26-b0aec2019720\") " pod="kube-system/konnectivity-agent-vgdrk"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757536    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-tuned\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:18.757550    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757599    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd2bec1a-032b-42cd-9555-7154268c805c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:18.757650    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret podName:64edf751-2ca9-432a-a626-2c23a5f79032 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:19.257607448 +0000 UTC m=+3.064018681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret") pod "global-pull-secret-syncer-l8zzv" (UID: "64edf751-2ca9-432a-a626-2c23a5f79032") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757695    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cd2bec1a-032b-42cd-9555-7154268c805c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757708    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757733    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-env-overrides\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757776    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd2bec1a-032b-42cd-9555-7154268c805c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757779    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-os-release\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757837    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-run-k8s-cni-cncf-io\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757844    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-os-release\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757881    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd2bec1a-032b-42cd-9555-7154268c805c-cnibin\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757922    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-sys-fs\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:18.757553    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:18.758518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757959    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-etc-openvswitch\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757992    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd2bec1a-032b-42cd-9555-7154268c805c-system-cni-dir\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:18.758027    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs podName:7c65b730-bf89-4090-b8a7-cad9a31bf810 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:19.257997345 +0000 UTC m=+3.064408559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs") pod "network-metrics-daemon-6dn2c" (UID: "7c65b730-bf89-4090-b8a7-cad9a31bf810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757948    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/410e2099-4588-45d5-a93c-cb4066032492-serviceca\") pod \"node-ca-hsf6v\" (UID: \"410e2099-4588-45d5-a93c-cb4066032492\") " pod="openshift-image-registry/node-ca-hsf6v"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758121    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd2bec1a-032b-42cd-9555-7154268c805c-cnibin\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757887    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-run-k8s-cni-cncf-io\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758210    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-etc-openvswitch\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758212    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-ovn-node-metrics-cert\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.757995    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/199bdf35-3af4-4b29-8e0d-1c68fc3ec572-iptables-alerter-script\") pod \"iptables-alerter-jksgp\" (UID: \"199bdf35-3af4-4b29-8e0d-1c68fc3ec572\") " pod="openshift-network-operator/iptables-alerter-jksgp"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758278    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/410e2099-4588-45d5-a93c-cb4066032492-host\") pod \"node-ca-hsf6v\" (UID: \"410e2099-4588-45d5-a93c-cb4066032492\") " pod="openshift-image-registry/node-ca-hsf6v"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758310    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd2bec1a-032b-42cd-9555-7154268c805c-os-release\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758339    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr2z4\" (UniqueName: \"kubernetes.io/projected/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-kube-api-access-lr2z4\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758366    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-env-overrides\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758368    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-run\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758440    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-run\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758444    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-sys\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758485    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zdgs\" (UniqueName: \"kubernetes.io/projected/7c65b730-bf89-4090-b8a7-cad9a31bf810-kube-api-access-7zdgs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:18.759421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758495    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-sys\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758516    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-run-systemd\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758549    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6d6l\" (UniqueName: \"kubernetes.io/projected/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-kube-api-access-g6d6l\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758577    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-multus-socket-dir-parent\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758601    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-multus-conf-dir\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758631    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-run-multus-certs\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758660    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-sysctl-d\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758688    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-slash\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758717    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-cnibin\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758740    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b4e497b3-17fb-4e64-bd26-b0aec2019720-agent-certs\") pod \"konnectivity-agent-vgdrk\" (UID: \"b4e497b3-17fb-4e64-bd26-b0aec2019720\") " pod="kube-system/konnectivity-agent-vgdrk"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758748    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cd2bec1a-032b-42cd-9555-7154268c805c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758770    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gz67\" (UniqueName: \"kubernetes.io/projected/199bdf35-3af4-4b29-8e0d-1c68fc3ec572-kube-api-access-8gz67\") pod \"iptables-alerter-jksgp\" (UID: \"199bdf35-3af4-4b29-8e0d-1c68fc3ec572\") " pod="openshift-network-operator/iptables-alerter-jksgp"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758816    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4fdfd2d4-f612-4ee0-b4cd-66417c138705-hosts-file\") pod \"node-resolver-5z66q\" (UID: \"4fdfd2d4-f612-4ee0-b4cd-66417c138705\") " pod="openshift-dns/node-resolver-5z66q"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758846    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd2bec1a-032b-42cd-9555-7154268c805c-cni-binary-copy\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758880    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cd2bec1a-032b-42cd-9555-7154268c805c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.758915    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-lib-modules\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.759064    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/410e2099-4588-45d5-a93c-cb4066032492-host\") pod \"node-ca-hsf6v\" (UID: \"410e2099-4588-45d5-a93c-cb4066032492\") " pod="openshift-image-registry/node-ca-hsf6v"
Jul 23 00:23:18.760252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.759081    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-lib-modules\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.759450    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd2bec1a-032b-42cd-9555-7154268c805c-os-release\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.759508    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4fdfd2d4-f612-4ee0-b4cd-66417c138705-hosts-file\") pod \"node-resolver-5z66q\" (UID: \"4fdfd2d4-f612-4ee0-b4cd-66417c138705\") " pod="openshift-dns/node-resolver-5z66q"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.759576    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-run-systemd\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.759676    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd2bec1a-032b-42cd-9555-7154268c805c-cni-binary-copy\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.759843    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-multus-socket-dir-parent\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.759897    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-multus-conf-dir\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.759945    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-host-run-multus-certs\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.760034    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7be69b0c-5f23-4e59-a484-022efa234e0e-etc-sysctl-d\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.760083    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-host-slash\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.760115    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cd2bec1a-032b-42cd-9555-7154268c805c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.760141    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db83e8c2-912c-42dc-8397-b428ebca65b8-cnibin\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.759964    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7be69b0c-5f23-4e59-a484-022efa234e0e-tmp\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.760871    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/199bdf35-3af4-4b29-8e0d-1c68fc3ec572-iptables-alerter-script\") pod \"iptables-alerter-jksgp\" (UID: \"199bdf35-3af4-4b29-8e0d-1c68fc3ec572\") " pod="openshift-network-operator/iptables-alerter-jksgp"
Jul 23 00:23:18.761099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.760882    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-ovnkube-script-lib\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.762176 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:18.762138    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:23:18.762277 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:18.762187    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:23:18.762277 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:18.762201    2561 projected.go:196] Error preparing data for projected volume kube-api-access-v9646 for pod openshift-network-diagnostics/network-check-target-nzvr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:18.762277 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:18.762264    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646 podName:c5eeb225-2f11-4dd2-b57f-933946195251 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:19.262247226 +0000 UTC m=+3.068658452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v9646" (UniqueName: "kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646") pod "network-check-target-nzvr2" (UID: "c5eeb225-2f11-4dd2-b57f-933946195251") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:18.764431 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.764252    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b4e497b3-17fb-4e64-bd26-b0aec2019720-agent-certs\") pod \"konnectivity-agent-vgdrk\" (UID: \"b4e497b3-17fb-4e64-bd26-b0aec2019720\") " pod="kube-system/konnectivity-agent-vgdrk"
Jul 23 00:23:18.766010 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.764969    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb7n2\" (UniqueName: \"kubernetes.io/projected/4fdfd2d4-f612-4ee0-b4cd-66417c138705-kube-api-access-cb7n2\") pod \"node-resolver-5z66q\" (UID: \"4fdfd2d4-f612-4ee0-b4cd-66417c138705\") " pod="openshift-dns/node-resolver-5z66q"
Jul 23 00:23:18.766010 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.765417    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmzxj\" (UniqueName: \"kubernetes.io/projected/7be69b0c-5f23-4e59-a484-022efa234e0e-kube-api-access-mmzxj\") pod \"tuned-drxfq\" (UID: \"7be69b0c-5f23-4e59-a484-022efa234e0e\") " pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.766010 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.765510    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzhdq\" (UniqueName: \"kubernetes.io/projected/cd2bec1a-032b-42cd-9555-7154268c805c-kube-api-access-bzhdq\") pod \"multus-additional-cni-plugins-b76l8\" (UID: \"cd2bec1a-032b-42cd-9555-7154268c805c\") " pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.766010 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.765965    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf7jg\" (UniqueName: \"kubernetes.io/projected/410e2099-4588-45d5-a93c-cb4066032492-kube-api-access-kf7jg\") pod \"node-ca-hsf6v\" (UID: \"410e2099-4588-45d5-a93c-cb4066032492\") " pod="openshift-image-registry/node-ca-hsf6v"
Jul 23 00:23:18.767650 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.767626    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f45wr\" (UniqueName: \"kubernetes.io/projected/db83e8c2-912c-42dc-8397-b428ebca65b8-kube-api-access-f45wr\") pod \"multus-zgm69\" (UID: \"db83e8c2-912c-42dc-8397-b428ebca65b8\") " pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.770243 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.769713    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zdgs\" (UniqueName: \"kubernetes.io/projected/7c65b730-bf89-4090-b8a7-cad9a31bf810-kube-api-access-7zdgs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:18.770243 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.769905    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gz67\" (UniqueName: \"kubernetes.io/projected/199bdf35-3af4-4b29-8e0d-1c68fc3ec572-kube-api-access-8gz67\") pod \"iptables-alerter-jksgp\" (UID: \"199bdf35-3af4-4b29-8e0d-1c68fc3ec572\") " pod="openshift-network-operator/iptables-alerter-jksgp"
Jul 23 00:23:18.771227 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.771203    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6d6l\" (UniqueName: \"kubernetes.io/projected/0c1097a5-8fea-43ca-be95-5f24ef1ebda4-kube-api-access-g6d6l\") pod \"ovnkube-node-g9g57\" (UID: \"0c1097a5-8fea-43ca-be95-5f24ef1ebda4\") " pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.786097 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.786019    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal" event={"ID":"dcbe9759263d750ea2db6eb7b4e9f691","Type":"ContainerStarted","Data":"409e9550a9173d203d24a174f3c4dc7452c503de1d85c6a835cf1ca154b9fcac"}
Jul 23 00:23:18.787292 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.787264    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal" event={"ID":"c98ef64082e55147c9ba40e982d64173","Type":"ContainerStarted","Data":"dc2b2d69bcaf2fbea0e69f52bd9523e19f11c11f9080338552a2402feae7e932"}
Jul 23 00:23:18.860202 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860104    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-sys-fs\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.860202 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860150    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr2z4\" (UniqueName: \"kubernetes.io/projected/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-kube-api-access-lr2z4\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.860202 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860191    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.860518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860215    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-socket-dir\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.860518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860243    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-etc-selinux\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.860518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860250    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-sys-fs\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.860518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860288    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-device-dir\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.860518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860306    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.860518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860369    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-device-dir\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.860518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860410    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-socket-dir\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.860518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860429    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-registration-dir\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.860518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860497    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-registration-dir\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.860518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.860502    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-etc-selinux\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.869478 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.869454    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr2z4\" (UniqueName: \"kubernetes.io/projected/5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d-kube-api-access-lr2z4\") pod \"aws-ebs-csi-driver-node-rr7c8\" (UID: \"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:18.941464 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.941431    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jksgp"
Jul 23 00:23:18.948381 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.948347    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hsf6v"
Jul 23 00:23:18.956261 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.956235    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zgm69"
Jul 23 00:23:18.961529 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.961474    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5z66q"
Jul 23 00:23:18.967995 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.967944    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b76l8"
Jul 23 00:23:18.974921 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.974900    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-drxfq"
Jul 23 00:23:18.981793 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.981772    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:18.992419 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.992395    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vgdrk"
Jul 23 00:23:18.998555 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:18.998533    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8"
Jul 23 00:23:19.033060 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:19.033017    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb83e8c2_912c_42dc_8397_b428ebca65b8.slice/crio-1d92c9558e6118cc9397a8e17afdf1b762a260381982ef25939d620d187a707a WatchSource:0}: Error finding container 1d92c9558e6118cc9397a8e17afdf1b762a260381982ef25939d620d187a707a: Status 404 returned error can't find the container with id 1d92c9558e6118cc9397a8e17afdf1b762a260381982ef25939d620d187a707a
Jul 23 00:23:19.034244 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:19.034204    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c1097a5_8fea_43ca_be95_5f24ef1ebda4.slice/crio-eb045b968cc121cd1900e01f01da9ed98bdfa5993fe33e0a9d76602b6bd0f65a WatchSource:0}: Error finding container eb045b968cc121cd1900e01f01da9ed98bdfa5993fe33e0a9d76602b6bd0f65a: Status 404 returned error can't find the container with id eb045b968cc121cd1900e01f01da9ed98bdfa5993fe33e0a9d76602b6bd0f65a
Jul 23 00:23:19.039767 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:19.039737    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fdfd2d4_f612_4ee0_b4cd_66417c138705.slice/crio-1c322b00c385bf66e4c7a871ec2c3ef3c820a20e2e3e058a6aad513344d76228 WatchSource:0}: Error finding container 1c322b00c385bf66e4c7a871ec2c3ef3c820a20e2e3e058a6aad513344d76228: Status 404 returned error can't find the container with id 1c322b00c385bf66e4c7a871ec2c3ef3c820a20e2e3e058a6aad513344d76228
Jul 23 00:23:19.042624 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:19.042601    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod410e2099_4588_45d5_a93c_cb4066032492.slice/crio-f71556f4f80353b38f66f1ea5876a1d2d5144851f30d0c99443d3e87b4cf5bed WatchSource:0}: Error finding container f71556f4f80353b38f66f1ea5876a1d2d5144851f30d0c99443d3e87b4cf5bed: Status 404 returned error can't find the container with id f71556f4f80353b38f66f1ea5876a1d2d5144851f30d0c99443d3e87b4cf5bed
Jul 23 00:23:19.043023 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:19.042987    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2bec1a_032b_42cd_9555_7154268c805c.slice/crio-ec2eb4be100a42e7c825c34965eae08ffd0a55e663e2955d1d95d272f66ba798 WatchSource:0}: Error finding container ec2eb4be100a42e7c825c34965eae08ffd0a55e663e2955d1d95d272f66ba798: Status 404 returned error can't find the container with id ec2eb4be100a42e7c825c34965eae08ffd0a55e663e2955d1d95d272f66ba798
Jul 23 00:23:19.045222 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:19.045008    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod199bdf35_3af4_4b29_8e0d_1c68fc3ec572.slice/crio-b8376cda7a7e24e45f3dd72f1df0e1d36edd66da9f5c3cef000e03593af88be8 WatchSource:0}: Error finding container b8376cda7a7e24e45f3dd72f1df0e1d36edd66da9f5c3cef000e03593af88be8: Status 404 returned error can't find the container with id b8376cda7a7e24e45f3dd72f1df0e1d36edd66da9f5c3cef000e03593af88be8
Jul 23 00:23:19.045427 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:19.045375    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e497b3_17fb_4e64_bd26_b0aec2019720.slice/crio-7cb420ec41e1d31d57564c3f4acbbccd1b04e97e1d5bac21a3b085a33c5e80d1 WatchSource:0}: Error finding container 7cb420ec41e1d31d57564c3f4acbbccd1b04e97e1d5bac21a3b085a33c5e80d1: Status 404 returned error can't find the container with id 7cb420ec41e1d31d57564c3f4acbbccd1b04e97e1d5bac21a3b085a33c5e80d1
Jul 23 00:23:19.264401 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.264350    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9646\" (UniqueName: \"kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646\") pod \"network-check-target-nzvr2\" (UID: \"c5eeb225-2f11-4dd2-b57f-933946195251\") " pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:19.264552 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.264437    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:19.264552 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.264465    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:19.264552 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:19.264530    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:23:19.264552 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:19.264551    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:23:19.264765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:19.264562    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:19.264765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:19.264585    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:19.264765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:19.264612    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs podName:7c65b730-bf89-4090-b8a7-cad9a31bf810 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:20.264599123 +0000 UTC m=+4.071010335 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs") pod "network-metrics-daemon-6dn2c" (UID: "7c65b730-bf89-4090-b8a7-cad9a31bf810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:19.264765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:19.264638    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret podName:64edf751-2ca9-432a-a626-2c23a5f79032 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:20.264625569 +0000 UTC m=+4.071036780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret") pod "global-pull-secret-syncer-l8zzv" (UID: "64edf751-2ca9-432a-a626-2c23a5f79032") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:19.264765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:19.264565    2561 projected.go:196] Error preparing data for projected volume kube-api-access-v9646 for pod openshift-network-diagnostics/network-check-target-nzvr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:19.264765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:19.264709    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646 podName:c5eeb225-2f11-4dd2-b57f-933946195251 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:20.264694922 +0000 UTC m=+4.071106138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v9646" (UniqueName: "kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646") pod "network-check-target-nzvr2" (UID: "c5eeb225-2f11-4dd2-b57f-933946195251") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:19.680911 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.680843    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 00:18:18 +0000 UTC" deadline="2028-04-17 11:27:42.344041108 +0000 UTC"
Jul 23 00:23:19.680911 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.680870    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15227h4m22.663174043s"
Jul 23 00:23:19.782572 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.782049    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:19.782572 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:19.782189    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:19.792344 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.792295    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5z66q" event={"ID":"4fdfd2d4-f612-4ee0-b4cd-66417c138705","Type":"ContainerStarted","Data":"1c322b00c385bf66e4c7a871ec2c3ef3c820a20e2e3e058a6aad513344d76228"}
Jul 23 00:23:19.798668 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.798637    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57" event={"ID":"0c1097a5-8fea-43ca-be95-5f24ef1ebda4","Type":"ContainerStarted","Data":"eb045b968cc121cd1900e01f01da9ed98bdfa5993fe33e0a9d76602b6bd0f65a"}
Jul 23 00:23:19.804997 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.804091    2561 generic.go:356] "Generic (PLEG): container finished" podID="c98ef64082e55147c9ba40e982d64173" containerID="c4076fafb5ec55428c5fbe0ab75927f258282a1489805e448f0ceb7bf89d751f" exitCode=0
Jul 23 00:23:19.804997 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.804233    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal" event={"ID":"c98ef64082e55147c9ba40e982d64173","Type":"ContainerDied","Data":"c4076fafb5ec55428c5fbe0ab75927f258282a1489805e448f0ceb7bf89d751f"}
Jul 23 00:23:19.813221 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.813193    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8" event={"ID":"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d","Type":"ContainerStarted","Data":"9fe2cdff8c6456242ba58fb2ec655e80ce0ecd6e4a16cd7421205fd11316e2d2"}
Jul 23 00:23:19.817955 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.815131    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vgdrk" event={"ID":"b4e497b3-17fb-4e64-bd26-b0aec2019720","Type":"ContainerStarted","Data":"7cb420ec41e1d31d57564c3f4acbbccd1b04e97e1d5bac21a3b085a33c5e80d1"}
Jul 23 00:23:19.823247 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.823220    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zgm69" event={"ID":"db83e8c2-912c-42dc-8397-b428ebca65b8","Type":"ContainerStarted","Data":"1d92c9558e6118cc9397a8e17afdf1b762a260381982ef25939d620d187a707a"}
Jul 23 00:23:19.824828 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.824803    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-drxfq" event={"ID":"7be69b0c-5f23-4e59-a484-022efa234e0e","Type":"ContainerStarted","Data":"3e96d04de08d9590fa5b6dfcb62c5f7e8af37e78af36302abb071bd725850f8d"}
Jul 23 00:23:19.826524 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.826471    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jksgp" event={"ID":"199bdf35-3af4-4b29-8e0d-1c68fc3ec572","Type":"ContainerStarted","Data":"b8376cda7a7e24e45f3dd72f1df0e1d36edd66da9f5c3cef000e03593af88be8"}
Jul 23 00:23:19.828121 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.828099    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b76l8" event={"ID":"cd2bec1a-032b-42cd-9555-7154268c805c","Type":"ContainerStarted","Data":"ec2eb4be100a42e7c825c34965eae08ffd0a55e663e2955d1d95d272f66ba798"}
Jul 23 00:23:19.829903 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:19.829880    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hsf6v" event={"ID":"410e2099-4588-45d5-a93c-cb4066032492","Type":"ContainerStarted","Data":"f71556f4f80353b38f66f1ea5876a1d2d5144851f30d0c99443d3e87b4cf5bed"}
Jul 23 00:23:20.275765 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:20.274410    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:20.275765 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:20.274690    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:20.275765 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:20.274761    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9646\" (UniqueName: \"kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646\") pod \"network-check-target-nzvr2\" (UID: \"c5eeb225-2f11-4dd2-b57f-933946195251\") " pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:20.275765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:20.274943    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:23:20.275765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:20.274964    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:23:20.275765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:20.274977    2561 projected.go:196] Error preparing data for projected volume kube-api-access-v9646 for pod openshift-network-diagnostics/network-check-target-nzvr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:20.275765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:20.275044    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646 podName:c5eeb225-2f11-4dd2-b57f-933946195251 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:22.275021897 +0000 UTC m=+6.081433112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v9646" (UniqueName: "kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646") pod "network-check-target-nzvr2" (UID: "c5eeb225-2f11-4dd2-b57f-933946195251") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:20.275765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:20.275479    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:20.275765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:20.275530    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs podName:7c65b730-bf89-4090-b8a7-cad9a31bf810 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:22.275513711 +0000 UTC m=+6.081924926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs") pod "network-metrics-daemon-6dn2c" (UID: "7c65b730-bf89-4090-b8a7-cad9a31bf810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:20.275765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:20.275593    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:20.275765 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:20.275627    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret podName:64edf751-2ca9-432a-a626-2c23a5f79032 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:22.275616293 +0000 UTC m=+6.082027506 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret") pod "global-pull-secret-syncer-l8zzv" (UID: "64edf751-2ca9-432a-a626-2c23a5f79032") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:20.520092 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:20.519883    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 00:23:20.681255 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:20.681162    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 00:18:18 +0000 UTC" deadline="2028-03-19 16:29:21.382011001 +0000 UTC"
Jul 23 00:23:20.681255 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:20.681219    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14536h6m0.700796564s"
Jul 23 00:23:20.784463 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:20.783779    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:20.784463 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:20.783908    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:20.784463 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:20.783988    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:20.784463 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:20.784061    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:20.843757 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:20.843700    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal" event={"ID":"c98ef64082e55147c9ba40e982d64173","Type":"ContainerStarted","Data":"2dba1d0f50fe578935a911634400d396f141b172e23f9ff3717ce35237ada4a3"}
Jul 23 00:23:20.848693 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:20.848655    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal" event={"ID":"dcbe9759263d750ea2db6eb7b4e9f691","Type":"ContainerStarted","Data":"f895199f3fc19df3c2f8d165702d7b8a90d26675ada5c737549750868413b784"}
Jul 23 00:23:20.875231 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:20.875159    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-215.ec2.internal" podStartSLOduration=2.875138064 podStartE2EDuration="2.875138064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:23:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:20.860691001 +0000 UTC m=+4.667102247" watchObservedRunningTime="2026-07-23 00:23:20.875138064 +0000 UTC m=+4.681549303"
Jul 23 00:23:21.781420 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:21.781387    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:21.781954 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:21.781534    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:22.291527 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:22.291479    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:22.291714 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:22.291594    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9646\" (UniqueName: \"kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646\") pod \"network-check-target-nzvr2\" (UID: \"c5eeb225-2f11-4dd2-b57f-933946195251\") " pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:22.291714 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:22.291648    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:22.291827 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:22.291767    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:22.291827 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:22.291827    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret podName:64edf751-2ca9-432a-a626-2c23a5f79032 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:26.291808862 +0000 UTC m=+10.098220076 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret") pod "global-pull-secret-syncer-l8zzv" (UID: "64edf751-2ca9-432a-a626-2c23a5f79032") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:22.292483 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:22.292202    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:22.292483 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:22.292257    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs podName:7c65b730-bf89-4090-b8a7-cad9a31bf810 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:26.292240299 +0000 UTC m=+10.098651546 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs") pod "network-metrics-daemon-6dn2c" (UID: "7c65b730-bf89-4090-b8a7-cad9a31bf810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:22.292483 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:22.292345    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:23:22.292483 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:22.292376    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:23:22.292483 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:22.292389    2561 projected.go:196] Error preparing data for projected volume kube-api-access-v9646 for pod openshift-network-diagnostics/network-check-target-nzvr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:22.292483 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:22.292428    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646 podName:c5eeb225-2f11-4dd2-b57f-933946195251 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:26.292413569 +0000 UTC m=+10.098824786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v9646" (UniqueName: "kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646") pod "network-check-target-nzvr2" (UID: "c5eeb225-2f11-4dd2-b57f-933946195251") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:22.781355 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:22.781267    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:22.781355 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:22.781318    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:22.781801 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:22.781440    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:22.781801 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:22.781543    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:23.782123 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:23.782077    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:23.782706 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:23.782258    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:24.783193 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:24.782495    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:24.783193 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:24.782634    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:24.783193 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:24.782994    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:24.783193 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:24.783100    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:25.781940 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:25.781902    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:25.782130 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:25.782040    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:26.326249 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:26.326207    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9646\" (UniqueName: \"kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646\") pod \"network-check-target-nzvr2\" (UID: \"c5eeb225-2f11-4dd2-b57f-933946195251\") " pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:26.326728 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:26.326281    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:26.326728 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:26.326308    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:26.326728 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:26.326461    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:26.326728 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:26.326469    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:26.326728 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:26.326482    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:23:26.326728 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:26.326530    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs podName:7c65b730-bf89-4090-b8a7-cad9a31bf810 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:34.326510469 +0000 UTC m=+18.132921694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs") pod "network-metrics-daemon-6dn2c" (UID: "7c65b730-bf89-4090-b8a7-cad9a31bf810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:26.326728 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:26.326542    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:23:26.326728 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:26.326552    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret podName:64edf751-2ca9-432a-a626-2c23a5f79032 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:34.326542575 +0000 UTC m=+18.132953785 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret") pod "global-pull-secret-syncer-l8zzv" (UID: "64edf751-2ca9-432a-a626-2c23a5f79032") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:26.326728 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:26.326556    2561 projected.go:196] Error preparing data for projected volume kube-api-access-v9646 for pod openshift-network-diagnostics/network-check-target-nzvr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:26.326728 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:26.326605    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646 podName:c5eeb225-2f11-4dd2-b57f-933946195251 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:34.326589012 +0000 UTC m=+18.133000224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v9646" (UniqueName: "kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646") pod "network-check-target-nzvr2" (UID: "c5eeb225-2f11-4dd2-b57f-933946195251") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:26.783043 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:26.782966    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:26.783206 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:26.783086    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:26.783546 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:26.783382    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:26.783546 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:26.783494    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:27.782136 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:27.782054    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:27.782592 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:27.782209    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:28.782159 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:28.782129    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:28.782659 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:28.782263    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:28.782659 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:28.782290    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:28.782659 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:28.782387    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:29.782107 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:29.782070    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:29.782258 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:29.782183    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:30.781230 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:30.781201    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:30.781402 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:30.781336    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:30.781449 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:30.781414    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:30.781522 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:30.781501    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:31.781473 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:31.781437    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:31.781924 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:31.781588    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:32.781581 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:32.781497    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:32.782067 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:32.781637    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:32.782067 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:32.781698    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:32.782067 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:32.781810    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:33.781416 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:33.781320    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:33.781608 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:33.781459    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:34.386134 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:34.386099    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:34.386335 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:34.386158    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9646\" (UniqueName: \"kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646\") pod \"network-check-target-nzvr2\" (UID: \"c5eeb225-2f11-4dd2-b57f-933946195251\") " pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:34.386335 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:34.386195    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:34.386335 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:34.386278    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:34.386335 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:34.386339    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret podName:64edf751-2ca9-432a-a626-2c23a5f79032 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:50.386320762 +0000 UTC m=+34.192731973 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret") pod "global-pull-secret-syncer-l8zzv" (UID: "64edf751-2ca9-432a-a626-2c23a5f79032") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:34.386540 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:34.386342    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:23:34.386540 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:34.386275    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:34.386540 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:34.386383    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:23:34.386540 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:34.386397    2561 projected.go:196] Error preparing data for projected volume kube-api-access-v9646 for pod openshift-network-diagnostics/network-check-target-nzvr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:34.386540 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:34.386445    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs podName:7c65b730-bf89-4090-b8a7-cad9a31bf810 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:50.386418726 +0000 UTC m=+34.192829941 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs") pod "network-metrics-daemon-6dn2c" (UID: "7c65b730-bf89-4090-b8a7-cad9a31bf810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:34.386540 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:34.386463    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646 podName:c5eeb225-2f11-4dd2-b57f-933946195251 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:50.386454161 +0000 UTC m=+34.192865377 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v9646" (UniqueName: "kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646") pod "network-check-target-nzvr2" (UID: "c5eeb225-2f11-4dd2-b57f-933946195251") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:34.782080 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:34.782000    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:34.782537 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:34.782136    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:34.782537 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:34.782274    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:34.782537 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:34.782398    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:35.781554 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:35.781523    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:35.781717 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:35.781650    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:36.782561 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:36.782535    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:36.782988 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:36.782604    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:36.782988 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:36.782702    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:36.782988 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:36.782836    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:37.782114 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:37.782047    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:37.782275 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:37.782190    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:38.782226 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.782042    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:38.782816 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.782097    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:38.782816 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:38.782342    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:38.782816 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:38.782475    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:38.888389 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.888168    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-drxfq" event={"ID":"7be69b0c-5f23-4e59-a484-022efa234e0e","Type":"ContainerStarted","Data":"877651aa894044edf2eb1cf6b6f8bd79c0211b7c571600a6ee5b63ef01cfeaaa"}
Jul 23 00:23:38.889870 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.889846    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd2bec1a-032b-42cd-9555-7154268c805c" containerID="3b8f03b168fdd747838e474416a9ec45fcfcaed3789a3cfc064470eba0296293" exitCode=0
Jul 23 00:23:38.889985 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.889918    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b76l8" event={"ID":"cd2bec1a-032b-42cd-9555-7154268c805c","Type":"ContainerDied","Data":"3b8f03b168fdd747838e474416a9ec45fcfcaed3789a3cfc064470eba0296293"}
Jul 23 00:23:38.891536 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.891512    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hsf6v" event={"ID":"410e2099-4588-45d5-a93c-cb4066032492","Type":"ContainerStarted","Data":"5d20f2d9e08868d7eeb1b99f89b252a15c312621ec91d72b2974d08586f26ac3"}
Jul 23 00:23:38.893221 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.893194    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5z66q" event={"ID":"4fdfd2d4-f612-4ee0-b4cd-66417c138705","Type":"ContainerStarted","Data":"4d4422b45aabdc0cc53d28be4b4584caa4954b7dc7d58acc27569d727d5cfb76"}
Jul 23 00:23:38.895530 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.895512    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:23:38.895899 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.895875    2561 generic.go:356] "Generic (PLEG): container finished" podID="0c1097a5-8fea-43ca-be95-5f24ef1ebda4" containerID="f85b77df7b8039bbca58913c0ac293682386780733242a3804762f7bb64ba871" exitCode=1
Jul 23 00:23:38.896004 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.895934    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57" event={"ID":"0c1097a5-8fea-43ca-be95-5f24ef1ebda4","Type":"ContainerStarted","Data":"ad326f74d63a907aba3c69b169555c203d446fc29abe6c856c58643880449efd"}
Jul 23 00:23:38.896086 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.896011    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57" event={"ID":"0c1097a5-8fea-43ca-be95-5f24ef1ebda4","Type":"ContainerStarted","Data":"1612b38c84f1e5e4b0df18aa8f28d84c8d77fe7f143695838e1784ac0b02edea"}
Jul 23 00:23:38.896086 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.896062    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57" event={"ID":"0c1097a5-8fea-43ca-be95-5f24ef1ebda4","Type":"ContainerDied","Data":"f85b77df7b8039bbca58913c0ac293682386780733242a3804762f7bb64ba871"}
Jul 23 00:23:38.896186 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.896115    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57" event={"ID":"0c1097a5-8fea-43ca-be95-5f24ef1ebda4","Type":"ContainerStarted","Data":"ca39107dc2bacd92ac76f042cee3ffa6d13c17ba57c36a0c8cb11b295a76b7d2"}
Jul 23 00:23:38.899168 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.899148    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8" event={"ID":"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d","Type":"ContainerStarted","Data":"3263cc853af642d6b9c3a3e3bf72d16575f24899ddef61fa17d03858c00b2c80"}
Jul 23 00:23:38.900514 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.900492    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vgdrk" event={"ID":"b4e497b3-17fb-4e64-bd26-b0aec2019720","Type":"ContainerStarted","Data":"68c6ea2e000721c199d00fe3a0167dfd02698a4482e58afa00bad1ed73664607"}
Jul 23 00:23:38.901726 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.901703    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zgm69" event={"ID":"db83e8c2-912c-42dc-8397-b428ebca65b8","Type":"ContainerStarted","Data":"2ab34033fe922f3d1ac6e3a56324089519b2bc908af35e99b022c469a6807916"}
Jul 23 00:23:38.906072 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.906029    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-215.ec2.internal" podStartSLOduration=20.906013963 podStartE2EDuration="20.906013963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:23:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:20.8755067 +0000 UTC m=+4.681917934" watchObservedRunningTime="2026-07-23 00:23:38.906013963 +0000 UTC m=+22.712425202"
Jul 23 00:23:38.906513 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.906489    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-drxfq" podStartSLOduration=2.892763594 podStartE2EDuration="21.906482519s" totalImagesPullingTime="19.013718925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:38.90598191 +0000 UTC m=+22.712393143" watchObservedRunningTime="2026-07-23 00:23:38.906482519 +0000 UTC m=+22.712893751"
Jul 23 00:23:38.920246 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.920212    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5z66q" podStartSLOduration=2.896108824 podStartE2EDuration="21.920200203s" totalImagesPullingTime="19.024091379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:38.91994587 +0000 UTC m=+22.726357105" watchObservedRunningTime="2026-07-23 00:23:38.920200203 +0000 UTC m=+22.726611436"
Jul 23 00:23:38.943629 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.943588    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hsf6v" podStartSLOduration=3.924557314 podStartE2EDuration="22.943574227s" totalImagesPullingTime="19.019016913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:23:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:38.943289014 +0000 UTC m=+22.749700247" watchObservedRunningTime="2026-07-23 00:23:38.943574227 +0000 UTC m=+22.749985460"
Jul 23 00:23:38.988314 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:38.988272    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vgdrk" podStartSLOduration=7.641435517 podStartE2EDuration="21.988257151s" totalImagesPullingTime="14.346821634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:38.987991813 +0000 UTC m=+22.794403046" watchObservedRunningTime="2026-07-23 00:23:38.988257151 +0000 UTC m=+22.794668383"
Jul 23 00:23:39.009472 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:39.009420    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zgm69" podStartSLOduration=2.965905636 podStartE2EDuration="22.009404561s" totalImagesPullingTime="19.043498925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:39.006708408 +0000 UTC m=+22.813119642" watchObservedRunningTime="2026-07-23 00:23:39.009404561 +0000 UTC m=+22.815815793"
Jul 23 00:23:39.627375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:39.627333    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vgdrk"
Jul 23 00:23:39.628045 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:39.628029    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vgdrk"
Jul 23 00:23:39.781749 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:39.781717    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:39.782031 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:39.781861    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:39.905523 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:39.905447    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jksgp" event={"ID":"199bdf35-3af4-4b29-8e0d-1c68fc3ec572","Type":"ContainerStarted","Data":"118cb04c001b9282bb82f10287ee6e24f37df97883b75a2d76c42daa41b6b745"}
Jul 23 00:23:39.908670 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:39.908643    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:23:39.909008 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:39.908984    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57" event={"ID":"0c1097a5-8fea-43ca-be95-5f24ef1ebda4","Type":"ContainerStarted","Data":"35b52295d61c9d6fd384e87c6556a14465f279a36dec37c39607568ee73c71e2"}
Jul 23 00:23:39.909086 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:39.909067    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57" event={"ID":"0c1097a5-8fea-43ca-be95-5f24ef1ebda4","Type":"ContainerStarted","Data":"bfab0ed55ce2d13e076b1a64e22497211e9841796ed694660380474d88380f37"}
Jul 23 00:23:39.909772 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:39.909310    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vgdrk"
Jul 23 00:23:39.909901 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:39.909882    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vgdrk"
Jul 23 00:23:39.921283 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:39.921236    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jksgp" podStartSLOduration=9.573817816 podStartE2EDuration="23.921219577s" totalImagesPullingTime="14.347401761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:23:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:39.920613723 +0000 UTC m=+23.727024958" watchObservedRunningTime="2026-07-23 00:23:39.921219577 +0000 UTC m=+23.727630810"
Jul 23 00:23:40.278740 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:40.278643    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 00:23:40.714322 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:40.714214    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T00:23:40.278659197Z","UUID":"9625e73d-4141-4882-b267-069ee65a379f","Handler":null,"Name":"","Endpoint":""}
Jul 23 00:23:40.716257 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:40.716235    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 00:23:40.716413 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:40.716278    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 00:23:40.782208 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:40.782173    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:40.782399 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:40.782185    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:40.782399 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:40.782312    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:40.782518 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:40.782392    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:40.913807 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:40.913769    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8" event={"ID":"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d","Type":"ContainerStarted","Data":"9372b519fe816290bae50e27ca02f51bed46e851f4a9817176f4b614fcf0716c"}
Jul 23 00:23:41.781945 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:41.781915    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:41.782055 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:41.782024    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:41.919799 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:41.919769    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:23:41.920254 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:41.920188    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57" event={"ID":"0c1097a5-8fea-43ca-be95-5f24ef1ebda4","Type":"ContainerStarted","Data":"a701b93fa072b21ab212233e59cd3ca511f35b84d1d19b01ac4da416dacfb10b"}
Jul 23 00:23:41.922528 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:41.922498    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8" event={"ID":"5ce0f3fd-6d7d-4f08-8a82-f0e410c4692d","Type":"ContainerStarted","Data":"7415d57ce45b07741c947820a9e25849bac574b581cdcaaeda50c00bc4ce0f2e"}
Jul 23 00:23:41.945517 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:41.945416    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rr7c8" podStartSLOduration=2.692716967 podStartE2EDuration="24.945395237s" totalImagesPullingTime="22.25267827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:23:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:41.944695407 +0000 UTC m=+25.751106642" watchObservedRunningTime="2026-07-23 00:23:41.945395237 +0000 UTC m=+25.751806473"
Jul 23 00:23:42.781882 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:42.781678    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:42.782070 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:42.781708    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:42.782070 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:42.781989    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:42.782189 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:42.782060    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:43.781827 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:43.781798    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:43.782481 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:43.781909    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:43.928731 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:43.928654    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd2bec1a-032b-42cd-9555-7154268c805c" containerID="9b207e87f5ec76113a776926434af37d90d981fecc7b0f6f56bb51b3c0d10299" exitCode=0
Jul 23 00:23:43.928890 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:43.928741    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b76l8" event={"ID":"cd2bec1a-032b-42cd-9555-7154268c805c","Type":"ContainerDied","Data":"9b207e87f5ec76113a776926434af37d90d981fecc7b0f6f56bb51b3c0d10299"}
Jul 23 00:23:43.932113 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:43.932095    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:23:43.932498 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:43.932471    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57" event={"ID":"0c1097a5-8fea-43ca-be95-5f24ef1ebda4","Type":"ContainerStarted","Data":"33afb0b55d1d1d809fc0f69957f34aab2d1e3691dda09eae18af742d33b8d5b8"}
Jul 23 00:23:43.932738 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:43.932719    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:43.932815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:43.932781    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:43.932958 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:43.932942    2561 scope.go:117] "RemoveContainer" containerID="f85b77df7b8039bbca58913c0ac293682386780733242a3804762f7bb64ba871"
Jul 23 00:23:43.949399 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:43.949347    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:44.782576 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:44.782301    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:44.783092 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:44.782644    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:44.783092 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:44.782316    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:44.783222 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:44.783101    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:44.939033 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:44.939003    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:23:44.939441 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:44.939416    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57" event={"ID":"0c1097a5-8fea-43ca-be95-5f24ef1ebda4","Type":"ContainerStarted","Data":"986303a723995ed5feacf49063a390fde97b642b0e09f640104d9d7ca10cd5d7"}
Jul 23 00:23:44.939765 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:44.939740    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:44.956720 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:44.956695    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:23:44.972444 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:44.972384    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57" podStartSLOduration=8.559962991 podStartE2EDuration="27.972350723s" totalImagesPullingTime="19.412387732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:44.970616326 +0000 UTC m=+28.777027571" watchObservedRunningTime="2026-07-23 00:23:44.972350723 +0000 UTC m=+28.778761961"
Jul 23 00:23:45.781976 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:45.781943    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:45.782225 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:45.782087    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:45.794723 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:45.794693    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nzvr2"]
Jul 23 00:23:45.795125 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:45.794800    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:45.795125 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:45.794912    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:45.797621 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:45.797597    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l8zzv"]
Jul 23 00:23:45.797754 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:45.797665    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:45.797754 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:45.797729    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:45.806967 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:45.806943    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6dn2c"]
Jul 23 00:23:45.943527 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:45.943496    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd2bec1a-032b-42cd-9555-7154268c805c" containerID="72776dbad80900c594c918a7e03174756ae8d01c8b4b2cb9c294410d94e9cd88" exitCode=0
Jul 23 00:23:45.943670 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:45.943560    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:45.943670 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:45.943580    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b76l8" event={"ID":"cd2bec1a-032b-42cd-9555-7154268c805c","Type":"ContainerDied","Data":"72776dbad80900c594c918a7e03174756ae8d01c8b4b2cb9c294410d94e9cd88"}
Jul 23 00:23:45.943670 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:45.943664    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:47.782195 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:47.782160    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:47.782195 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:47.782196    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:47.782641 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:47.782199    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:47.782641 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:47.782288    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:47.782641 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:47.782332    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:47.782641 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:47.782433    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:47.950784 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:47.950691    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd2bec1a-032b-42cd-9555-7154268c805c" containerID="e082da9f36752eb0cab12f022ddb2d015e465025bfd6274877c11396d6fc0eae" exitCode=0
Jul 23 00:23:47.950784 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:47.950732    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b76l8" event={"ID":"cd2bec1a-032b-42cd-9555-7154268c805c","Type":"ContainerDied","Data":"e082da9f36752eb0cab12f022ddb2d015e465025bfd6274877c11396d6fc0eae"}
Jul 23 00:23:49.781773 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:49.781593    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:49.782250 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:49.781642    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:49.782250 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:49.781859    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:49.782250 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:49.781966    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:49.782250 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:49.781642    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:49.782250 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:49.782088    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:50.399906 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:50.399874    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9646\" (UniqueName: \"kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646\") pod \"network-check-target-nzvr2\" (UID: \"c5eeb225-2f11-4dd2-b57f-933946195251\") " pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:50.400092 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:50.399925    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:50.400092 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:50.400019    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:50.400092 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:50.400035    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:23:50.400092 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:50.400056    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:23:50.400092 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:50.400064    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret podName:64edf751-2ca9-432a-a626-2c23a5f79032 nodeName:}" failed. No retries permitted until 2026-07-23 00:24:22.400051555 +0000 UTC m=+66.206462769 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret") pod "global-pull-secret-syncer-l8zzv" (UID: "64edf751-2ca9-432a-a626-2c23a5f79032") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:23:50.400092 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:50.400065    2561 projected.go:196] Error preparing data for projected volume kube-api-access-v9646 for pod openshift-network-diagnostics/network-check-target-nzvr2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:50.400092 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:50.400090    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:50.400453 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:50.400100    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646 podName:c5eeb225-2f11-4dd2-b57f-933946195251 nodeName:}" failed. No retries permitted until 2026-07-23 00:24:22.400090237 +0000 UTC m=+66.206501466 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-v9646" (UniqueName: "kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646") pod "network-check-target-nzvr2" (UID: "c5eeb225-2f11-4dd2-b57f-933946195251") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:23:50.400453 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:50.400037    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:50.400453 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:50.400134    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs podName:7c65b730-bf89-4090-b8a7-cad9a31bf810 nodeName:}" failed. No retries permitted until 2026-07-23 00:24:22.400122744 +0000 UTC m=+66.206533959 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs") pod "network-metrics-daemon-6dn2c" (UID: "7c65b730-bf89-4090-b8a7-cad9a31bf810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:23:51.782116 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.782084    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:51.782558 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.782190    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:51.782558 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.782197    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:51.782558 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:51.782309    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nzvr2" podUID="c5eeb225-2f11-4dd2-b57f-933946195251"
Jul 23 00:23:51.782558 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:51.782193    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-l8zzv" podUID="64edf751-2ca9-432a-a626-2c23a5f79032"
Jul 23 00:23:51.782558 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:51.782382    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:23:51.913290 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.913261    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-215.ec2.internal" event="NodeReady"
Jul 23 00:23:51.913481 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.913403    2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 00:23:51.946568 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.946534    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"]
Jul 23 00:23:51.948311 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.948292    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:23:51.949778 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.949752    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-868df8cc95-gjff5"]
Jul 23 00:23:51.950923 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.950883    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tvzb2\""
Jul 23 00:23:51.950923 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.950914    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 23 00:23:51.951117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.950929    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 23 00:23:51.951785 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.951766    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:51.954347 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.954182    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 23 00:23:51.954347 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.954231    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 23 00:23:51.954347 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.954245    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6v7ht\""
Jul 23 00:23:51.954347 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.954292    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 23 00:23:51.961914 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.960564    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"]
Jul 23 00:23:51.965292 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.965209    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xn4kv"]
Jul 23 00:23:51.965891 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.965868    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 23 00:23:51.971004 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.970341    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jgpkp"]
Jul 23 00:23:51.971004 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.970693    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xn4kv"
Jul 23 00:23:51.972501 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.972482    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-868df8cc95-gjff5"]
Jul 23 00:23:51.972623 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.972579    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:51.973738 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.973711    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vdgqd\""
Jul 23 00:23:51.973738 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.973724    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 00:23:51.973881 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.973772    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 00:23:51.974204 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.974189    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 00:23:51.974948 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.974932    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 00:23:51.975019 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.974963    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 00:23:51.975244 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.975227    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fhdn8\""
Jul 23 00:23:51.977701 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.977677    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xn4kv"]
Jul 23 00:23:51.981226 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:51.981206    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jgpkp"]
Jul 23 00:23:52.113404 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113358    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b5feda6-e797-4219-9e8a-bc0f0a26909f-installation-pull-secrets\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.113404 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113408    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7srd\" (UniqueName: \"kubernetes.io/projected/4fbb34aa-fae2-48b7-bda5-57e0031af622-kube-api-access-w7srd\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:52.113621 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113431    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4fbb34aa-fae2-48b7-bda5-57e0031af622-config-volume\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:52.113621 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113449    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-certificates\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.113621 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113469    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.113621 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113535    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/197306dc-60be-49d8-854c-dba425eeb32c-cert\") pod \"ingress-canary-xn4kv\" (UID: \"197306dc-60be-49d8-854c-dba425eeb32c\") " pod="openshift-ingress-canary/ingress-canary-xn4kv"
Jul 23 00:23:52.113621 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113567    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fbb34aa-fae2-48b7-bda5-57e0031af622-tmp-dir\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:52.113864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113627    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b5feda6-e797-4219-9e8a-bc0f0a26909f-trusted-ca\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.113864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113682    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b5feda6-e797-4219-9e8a-bc0f0a26909f-image-registry-private-configuration\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.113864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113717    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5dbp\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-kube-api-access-f5dbp\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.113864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113747    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d908e88c-afcd-4b67-97ef-b36595506a36-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:23:52.113864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113772    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:23:52.113864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113797    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-bound-sa-token\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.113864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113819    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd46k\" (UniqueName: \"kubernetes.io/projected/197306dc-60be-49d8-854c-dba425eeb32c-kube-api-access-kd46k\") pod \"ingress-canary-xn4kv\" (UID: \"197306dc-60be-49d8-854c-dba425eeb32c\") " pod="openshift-ingress-canary/ingress-canary-xn4kv"
Jul 23 00:23:52.113864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113860    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b5feda6-e797-4219-9e8a-bc0f0a26909f-ca-trust-extracted\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.114171 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.113883    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:52.215176 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215149    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5dbp\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-kube-api-access-f5dbp\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.215305 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215182    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d908e88c-afcd-4b67-97ef-b36595506a36-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:23:52.215305 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215198    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:23:52.215305 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215214    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-bound-sa-token\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.215305 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215231    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd46k\" (UniqueName: \"kubernetes.io/projected/197306dc-60be-49d8-854c-dba425eeb32c-kube-api-access-kd46k\") pod \"ingress-canary-xn4kv\" (UID: \"197306dc-60be-49d8-854c-dba425eeb32c\") " pod="openshift-ingress-canary/ingress-canary-xn4kv"
Jul 23 00:23:52.215305 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215258    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b5feda6-e797-4219-9e8a-bc0f0a26909f-ca-trust-extracted\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.215578 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.215322    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 00:23:52.215578 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.215418    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert podName:d908e88c-afcd-4b67-97ef-b36595506a36 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:52.715393996 +0000 UTC m=+36.521805228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dsmtv" (UID: "d908e88c-afcd-4b67-97ef-b36595506a36") : secret "networking-console-plugin-cert" not found
Jul 23 00:23:52.215578 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215538    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:52.215760 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215593    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b5feda6-e797-4219-9e8a-bc0f0a26909f-installation-pull-secrets\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.215760 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215617    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7srd\" (UniqueName: \"kubernetes.io/projected/4fbb34aa-fae2-48b7-bda5-57e0031af622-kube-api-access-w7srd\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:52.215760 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215628    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b5feda6-e797-4219-9e8a-bc0f0a26909f-ca-trust-extracted\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.215760 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215664    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4fbb34aa-fae2-48b7-bda5-57e0031af622-config-volume\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:52.215760 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.215755    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:23:52.216027 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.215806    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls podName:4fbb34aa-fae2-48b7-bda5-57e0031af622 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:52.715789103 +0000 UTC m=+36.522200314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls") pod "dns-default-jgpkp" (UID: "4fbb34aa-fae2-48b7-bda5-57e0031af622") : secret "dns-default-metrics-tls" not found
Jul 23 00:23:52.216027 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215854    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-certificates\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.216027 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215926    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.216027 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.215958    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/197306dc-60be-49d8-854c-dba425eeb32c-cert\") pod \"ingress-canary-xn4kv\" (UID: \"197306dc-60be-49d8-854c-dba425eeb32c\") " pod="openshift-ingress-canary/ingress-canary-xn4kv"
Jul 23 00:23:52.216027 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.216008    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fbb34aa-fae2-48b7-bda5-57e0031af622-tmp-dir\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:52.216027 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.216021    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d908e88c-afcd-4b67-97ef-b36595506a36-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:23:52.216320 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.216080    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b5feda6-e797-4219-9e8a-bc0f0a26909f-trusted-ca\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.216320 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.216150    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b5feda6-e797-4219-9e8a-bc0f0a26909f-image-registry-private-configuration\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.216320 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.216240    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4fbb34aa-fae2-48b7-bda5-57e0031af622-config-volume\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:52.216320 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.216308    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:23:52.216320 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.216323    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868df8cc95-gjff5: secret "image-registry-tls" not found
Jul 23 00:23:52.216594 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.216345    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fbb34aa-fae2-48b7-bda5-57e0031af622-tmp-dir\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:52.216594 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.216396    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls podName:6b5feda6-e797-4219-9e8a-bc0f0a26909f nodeName:}" failed. No retries permitted until 2026-07-23 00:23:52.716377973 +0000 UTC m=+36.522789189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls") pod "image-registry-868df8cc95-gjff5" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f") : secret "image-registry-tls" not found
Jul 23 00:23:52.216594 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.216552    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-certificates\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.217305 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.217264    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b5feda6-e797-4219-9e8a-bc0f0a26909f-trusted-ca\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.220281 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.220257    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b5feda6-e797-4219-9e8a-bc0f0a26909f-installation-pull-secrets\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.220386 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.220265    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b5feda6-e797-4219-9e8a-bc0f0a26909f-image-registry-private-configuration\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.220386 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.220293    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/197306dc-60be-49d8-854c-dba425eeb32c-cert\") pod \"ingress-canary-xn4kv\" (UID: \"197306dc-60be-49d8-854c-dba425eeb32c\") " pod="openshift-ingress-canary/ingress-canary-xn4kv"
Jul 23 00:23:52.224513 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.224490    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7srd\" (UniqueName: \"kubernetes.io/projected/4fbb34aa-fae2-48b7-bda5-57e0031af622-kube-api-access-w7srd\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:52.224608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.224575    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5dbp\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-kube-api-access-f5dbp\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.224935 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.224916    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-bound-sa-token\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.225062 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.225044    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd46k\" (UniqueName: \"kubernetes.io/projected/197306dc-60be-49d8-854c-dba425eeb32c-kube-api-access-kd46k\") pod \"ingress-canary-xn4kv\" (UID: \"197306dc-60be-49d8-854c-dba425eeb32c\") " pod="openshift-ingress-canary/ingress-canary-xn4kv"
Jul 23 00:23:52.282595 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.282560    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xn4kv"
Jul 23 00:23:52.410943 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.410915    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xn4kv"]
Jul 23 00:23:52.414893 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:52.414744    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod197306dc_60be_49d8_854c_dba425eeb32c.slice/crio-3d0849ba15790c9eae86edeba4323e697026a08637cbd9f0a8a4c67eca5179e3 WatchSource:0}: Error finding container 3d0849ba15790c9eae86edeba4323e697026a08637cbd9f0a8a4c67eca5179e3: Status 404 returned error can't find the container with id 3d0849ba15790c9eae86edeba4323e697026a08637cbd9f0a8a4c67eca5179e3
Jul 23 00:23:52.719945 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.719846    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:52.719945 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.719895    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:52.720150 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.719959    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:23:52.720150 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.719989    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:23:52.720150 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.720052    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls podName:4fbb34aa-fae2-48b7-bda5-57e0031af622 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:53.720037339 +0000 UTC m=+37.526448551 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls") pod "dns-default-jgpkp" (UID: "4fbb34aa-fae2-48b7-bda5-57e0031af622") : secret "dns-default-metrics-tls" not found
Jul 23 00:23:52.720150 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.720053    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 00:23:52.720150 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.720057    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:23:52.720150 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.720122    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868df8cc95-gjff5: secret "image-registry-tls" not found
Jul 23 00:23:52.720359 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.720100    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert podName:d908e88c-afcd-4b67-97ef-b36595506a36 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:53.720088774 +0000 UTC m=+37.526499985 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dsmtv" (UID: "d908e88c-afcd-4b67-97ef-b36595506a36") : secret "networking-console-plugin-cert" not found
Jul 23 00:23:52.720359 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:52.720189    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls podName:6b5feda6-e797-4219-9e8a-bc0f0a26909f nodeName:}" failed. No retries permitted until 2026-07-23 00:23:53.72017087 +0000 UTC m=+37.526582094 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls") pod "image-registry-868df8cc95-gjff5" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f") : secret "image-registry-tls" not found
Jul 23 00:23:52.973157 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.973074    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd2bec1a-032b-42cd-9555-7154268c805c" containerID="c9a30b8dd958a9e75da60220316471c3111b504d81fe76371e3b91fcf6d34943" exitCode=0
Jul 23 00:23:52.973778 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.973166    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b76l8" event={"ID":"cd2bec1a-032b-42cd-9555-7154268c805c","Type":"ContainerDied","Data":"c9a30b8dd958a9e75da60220316471c3111b504d81fe76371e3b91fcf6d34943"}
Jul 23 00:23:52.974352 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:52.974323    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xn4kv" event={"ID":"197306dc-60be-49d8-854c-dba425eeb32c","Type":"ContainerStarted","Data":"3d0849ba15790c9eae86edeba4323e697026a08637cbd9f0a8a4c67eca5179e3"}
Jul 23 00:23:53.728162 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.728125    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:23:53.728162 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.728165    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:53.728409 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.728200    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:53.728409 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:53.728285    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:23:53.728409 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:53.728290    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:23:53.728409 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:53.728351    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls podName:4fbb34aa-fae2-48b7-bda5-57e0031af622 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:55.728334817 +0000 UTC m=+39.534746028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls") pod "dns-default-jgpkp" (UID: "4fbb34aa-fae2-48b7-bda5-57e0031af622") : secret "dns-default-metrics-tls" not found
Jul 23 00:23:53.728409 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:53.728289    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 00:23:53.728569 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:53.728418    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert podName:d908e88c-afcd-4b67-97ef-b36595506a36 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:55.728407185 +0000 UTC m=+39.534818396 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dsmtv" (UID: "d908e88c-afcd-4b67-97ef-b36595506a36") : secret "networking-console-plugin-cert" not found
Jul 23 00:23:53.728569 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:53.728295    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868df8cc95-gjff5: secret "image-registry-tls" not found
Jul 23 00:23:53.728569 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:53.728445    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls podName:6b5feda6-e797-4219-9e8a-bc0f0a26909f nodeName:}" failed. No retries permitted until 2026-07-23 00:23:55.728439569 +0000 UTC m=+39.534850781 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls") pod "image-registry-868df8cc95-gjff5" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f") : secret "image-registry-tls" not found
Jul 23 00:23:53.781733 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.781669    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:23:53.781733 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.781705    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:23:53.781733 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.781708    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:23:53.784550 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.784528    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 00:23:53.784550 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.784538    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 00:23:53.785751 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.785554    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 00:23:53.785751 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.785638    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c75lg\""
Jul 23 00:23:53.785751 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.785673    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 00:23:53.785751 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.785559    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8gvmb\""
Jul 23 00:23:53.979528 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.979447    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd2bec1a-032b-42cd-9555-7154268c805c" containerID="b446ea892c732274fede3bb4adab9159a1e40b1507cfeac54ca735af28b994ac" exitCode=0
Jul 23 00:23:53.979528 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:53.979484    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b76l8" event={"ID":"cd2bec1a-032b-42cd-9555-7154268c805c","Type":"ContainerDied","Data":"b446ea892c732274fede3bb4adab9159a1e40b1507cfeac54ca735af28b994ac"}
Jul 23 00:23:54.987062 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:54.986864    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b76l8" event={"ID":"cd2bec1a-032b-42cd-9555-7154268c805c","Type":"ContainerStarted","Data":"bf06846a3137f836c34d196dc86bbb105a1a7e6937154ef00397a161b8e90336"}
Jul 23 00:23:55.010748 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:55.010702    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b76l8" podStartSLOduration=7.814415024 podStartE2EDuration="38.010686791s" totalImagesPullingTime="30.196271767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:23:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:55.010000555 +0000 UTC m=+38.816411788" watchObservedRunningTime="2026-07-23 00:23:55.010686791 +0000 UTC m=+38.817098002"
Jul 23 00:23:55.744890 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:55.744840    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:23:55.744890 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:55.744887    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:55.745141 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:55.744986    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:23:55.745141 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:55.744994    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 00:23:55.745141 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:55.745020    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:55.745141 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:55.745042    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls podName:4fbb34aa-fae2-48b7-bda5-57e0031af622 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:59.745027328 +0000 UTC m=+43.551438540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls") pod "dns-default-jgpkp" (UID: "4fbb34aa-fae2-48b7-bda5-57e0031af622") : secret "dns-default-metrics-tls" not found
Jul 23 00:23:55.745141 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:55.745084    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:23:55.745141 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:55.745093    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868df8cc95-gjff5: secret "image-registry-tls" not found
Jul 23 00:23:55.745141 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:55.745108    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert podName:d908e88c-afcd-4b67-97ef-b36595506a36 nodeName:}" failed. No retries permitted until 2026-07-23 00:23:59.745089718 +0000 UTC m=+43.551500943 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dsmtv" (UID: "d908e88c-afcd-4b67-97ef-b36595506a36") : secret "networking-console-plugin-cert" not found
Jul 23 00:23:55.745141 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:55.745139    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls podName:6b5feda6-e797-4219-9e8a-bc0f0a26909f nodeName:}" failed. No retries permitted until 2026-07-23 00:23:59.745124528 +0000 UTC m=+43.551535747 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls") pod "image-registry-868df8cc95-gjff5" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f") : secret "image-registry-tls" not found
Jul 23 00:23:55.991227 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:55.991190    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xn4kv" event={"ID":"197306dc-60be-49d8-854c-dba425eeb32c","Type":"ContainerStarted","Data":"6932eae52b1317b44dd73238d655996c6cdc3313bd4b79f6ec8853332fdc14aa"}
Jul 23 00:23:56.006290 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:56.006202    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xn4kv" podStartSLOduration=2.480625699 podStartE2EDuration="5.006185858s" totalImagesPullingTime="2.525560159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:56.005493717 +0000 UTC m=+39.811904950" watchObservedRunningTime="2026-07-23 00:23:56.006185858 +0000 UTC m=+39.812597091"
Jul 23 00:23:58.233743 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.233707    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k"]
Jul 23 00:23:58.265536 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.265509    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"]
Jul 23 00:23:58.265693 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.265662    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k"
Jul 23 00:23:58.271381 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.271347    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 23 00:23:58.272155 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.272130    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 23 00:23:58.272546 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.272526    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jmjnd\""
Jul 23 00:23:58.272546 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.272535    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 23 00:23:58.272691 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.272597    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 23 00:23:58.279836 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.279816    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"]
Jul 23 00:23:58.279944 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.279928    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:23:58.282831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.282806    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 23 00:23:58.291659 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.291643    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k"]
Jul 23 00:23:58.291747 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.291667    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"]
Jul 23 00:23:58.291747 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.291685    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"]
Jul 23 00:23:58.291817 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.291754    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.295785 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.295763    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 23 00:23:58.296833 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.296816    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 23 00:23:58.296927 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.296888    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 23 00:23:58.296927 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.296895    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 23 00:23:58.367037 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.367009    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqjtm\" (UniqueName: \"kubernetes.io/projected/2d9c04b3-b899-4884-9e89-0037f29f8a0f-kube-api-access-vqjtm\") pod \"managed-serviceaccount-addon-agent-6cb57d4b66-j974k\" (UID: \"2d9c04b3-b899-4884-9e89-0037f29f8a0f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k"
Jul 23 00:23:58.367212 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.367054    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d9c04b3-b899-4884-9e89-0037f29f8a0f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6cb57d4b66-j974k\" (UID: \"2d9c04b3-b899-4884-9e89-0037f29f8a0f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k"
Jul 23 00:23:58.367212 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.367121    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7dbt\" (UniqueName: \"kubernetes.io/projected/cd2d8df2-d752-4524-991f-aa1def67e57e-kube-api-access-z7dbt\") pod \"klusterlet-addon-workmgr-7c5ddd655b-ksf65\" (UID: \"cd2d8df2-d752-4524-991f-aa1def67e57e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:23:58.367212 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.367153    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd2d8df2-d752-4524-991f-aa1def67e57e-tmp\") pod \"klusterlet-addon-workmgr-7c5ddd655b-ksf65\" (UID: \"cd2d8df2-d752-4524-991f-aa1def67e57e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:23:58.367212 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.367187    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/cd2d8df2-d752-4524-991f-aa1def67e57e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c5ddd655b-ksf65\" (UID: \"cd2d8df2-d752-4524-991f-aa1def67e57e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:23:58.467985 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.467953    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqjtm\" (UniqueName: \"kubernetes.io/projected/2d9c04b3-b899-4884-9e89-0037f29f8a0f-kube-api-access-vqjtm\") pod \"managed-serviceaccount-addon-agent-6cb57d4b66-j974k\" (UID: \"2d9c04b3-b899-4884-9e89-0037f29f8a0f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k"
Jul 23 00:23:58.468111 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.467993    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.468111 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.468034    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd2d8df2-d752-4524-991f-aa1def67e57e-tmp\") pod \"klusterlet-addon-workmgr-7c5ddd655b-ksf65\" (UID: \"cd2d8df2-d752-4524-991f-aa1def67e57e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:23:58.468111 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.468089    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/cd2d8df2-d752-4524-991f-aa1def67e57e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c5ddd655b-ksf65\" (UID: \"cd2d8df2-d752-4524-991f-aa1def67e57e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:23:58.468272 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.468157    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d9c04b3-b899-4884-9e89-0037f29f8a0f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6cb57d4b66-j974k\" (UID: \"2d9c04b3-b899-4884-9e89-0037f29f8a0f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k"
Jul 23 00:23:58.468272 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.468189    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dbt\" (UniqueName: \"kubernetes.io/projected/cd2d8df2-d752-4524-991f-aa1def67e57e-kube-api-access-z7dbt\") pod \"klusterlet-addon-workmgr-7c5ddd655b-ksf65\" (UID: \"cd2d8df2-d752-4524-991f-aa1def67e57e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:23:58.468272 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.468229    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hddkw\" (UniqueName: \"kubernetes.io/projected/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-kube-api-access-hddkw\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.468272 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.468259    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-ca\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.468473 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.468291    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.468473 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.468341    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-hub\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.468473 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.468404    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.468473 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.468414    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd2d8df2-d752-4524-991f-aa1def67e57e-tmp\") pod \"klusterlet-addon-workmgr-7c5ddd655b-ksf65\" (UID: \"cd2d8df2-d752-4524-991f-aa1def67e57e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:23:58.472150 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.472123    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/cd2d8df2-d752-4524-991f-aa1def67e57e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c5ddd655b-ksf65\" (UID: \"cd2d8df2-d752-4524-991f-aa1def67e57e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:23:58.472252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.472235    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d9c04b3-b899-4884-9e89-0037f29f8a0f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6cb57d4b66-j974k\" (UID: \"2d9c04b3-b899-4884-9e89-0037f29f8a0f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k"
Jul 23 00:23:58.477158 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.477139    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7dbt\" (UniqueName: \"kubernetes.io/projected/cd2d8df2-d752-4524-991f-aa1def67e57e-kube-api-access-z7dbt\") pod \"klusterlet-addon-workmgr-7c5ddd655b-ksf65\" (UID: \"cd2d8df2-d752-4524-991f-aa1def67e57e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:23:58.477406 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.477356    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqjtm\" (UniqueName: \"kubernetes.io/projected/2d9c04b3-b899-4884-9e89-0037f29f8a0f-kube-api-access-vqjtm\") pod \"managed-serviceaccount-addon-agent-6cb57d4b66-j974k\" (UID: \"2d9c04b3-b899-4884-9e89-0037f29f8a0f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k"
Jul 23 00:23:58.569501 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.569405    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hddkw\" (UniqueName: \"kubernetes.io/projected/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-kube-api-access-hddkw\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.569501 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.569441    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-ca\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.569501 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.569460    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.569501 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.569477    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-hub\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.569808 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.569604    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.569808 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.569653    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.570385 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.570346    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.571968 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.571939    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.572078 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.571976    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-ca\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.572078 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.572009    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.572151 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.572075    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-hub\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.577100 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.577080    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hddkw\" (UniqueName: \"kubernetes.io/projected/d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e-kube-api-access-hddkw\") pod \"cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs\" (UID: \"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.586795 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.586776    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k"
Jul 23 00:23:58.594018 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.593998    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:23:58.600178 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.600156    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:23:58.748276 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.748238    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k"]
Jul 23 00:23:58.751748 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:58.751689    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d9c04b3_b899_4884_9e89_0037f29f8a0f.slice/crio-c4216294996010aca44b1d14dca14d749a35a19f49b2c330a7c0e8a96c150ec6 WatchSource:0}: Error finding container c4216294996010aca44b1d14dca14d749a35a19f49b2c330a7c0e8a96c150ec6: Status 404 returned error can't find the container with id c4216294996010aca44b1d14dca14d749a35a19f49b2c330a7c0e8a96c150ec6
Jul 23 00:23:58.771870 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.770584    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"]
Jul 23 00:23:58.776511 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:58.776487    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2d8df2_d752_4524_991f_aa1def67e57e.slice/crio-ccadcd88b7eed1a73e18d921b461a40b1c96e294563fec26be36dc4d52648281 WatchSource:0}: Error finding container ccadcd88b7eed1a73e18d921b461a40b1c96e294563fec26be36dc4d52648281: Status 404 returned error can't find the container with id ccadcd88b7eed1a73e18d921b461a40b1c96e294563fec26be36dc4d52648281
Jul 23 00:23:58.799674 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.799643    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"]
Jul 23 00:23:58.802275 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:23:58.802248    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5eb4f62_b8d0_48bf_aa4a_a2c48646dd0e.slice/crio-dc01a8d5a8df63751a2d38ae81209f36632c491fa61b55d56b8c75b412155b81 WatchSource:0}: Error finding container dc01a8d5a8df63751a2d38ae81209f36632c491fa61b55d56b8c75b412155b81: Status 404 returned error can't find the container with id dc01a8d5a8df63751a2d38ae81209f36632c491fa61b55d56b8c75b412155b81
Jul 23 00:23:58.998460 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.998420    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65" event={"ID":"cd2d8df2-d752-4524-991f-aa1def67e57e","Type":"ContainerStarted","Data":"ccadcd88b7eed1a73e18d921b461a40b1c96e294563fec26be36dc4d52648281"}
Jul 23 00:23:58.999508 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:58.999484    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k" event={"ID":"2d9c04b3-b899-4884-9e89-0037f29f8a0f","Type":"ContainerStarted","Data":"c4216294996010aca44b1d14dca14d749a35a19f49b2c330a7c0e8a96c150ec6"}
Jul 23 00:23:59.000454 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:59.000437    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs" event={"ID":"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e","Type":"ContainerStarted","Data":"dc01a8d5a8df63751a2d38ae81209f36632c491fa61b55d56b8c75b412155b81"}
Jul 23 00:23:59.784154 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:59.783059    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:23:59.784154 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:59.783184    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:23:59.784154 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:23:59.783217    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:23:59.784154 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:59.783384    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:23:59.784154 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:59.783450    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls podName:4fbb34aa-fae2-48b7-bda5-57e0031af622 nodeName:}" failed. No retries permitted until 2026-07-23 00:24:07.783430105 +0000 UTC m=+51.589841318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls") pod "dns-default-jgpkp" (UID: "4fbb34aa-fae2-48b7-bda5-57e0031af622") : secret "dns-default-metrics-tls" not found
Jul 23 00:23:59.784154 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:59.783918    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:23:59.784154 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:59.783935    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868df8cc95-gjff5: secret "image-registry-tls" not found
Jul 23 00:23:59.784154 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:59.783985    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls podName:6b5feda6-e797-4219-9e8a-bc0f0a26909f nodeName:}" failed. No retries permitted until 2026-07-23 00:24:07.783970044 +0000 UTC m=+51.590381258 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls") pod "image-registry-868df8cc95-gjff5" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f") : secret "image-registry-tls" not found
Jul 23 00:23:59.784154 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:59.784041    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 00:23:59.784154 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:23:59.784072    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert podName:d908e88c-afcd-4b67-97ef-b36595506a36 nodeName:}" failed. No retries permitted until 2026-07-23 00:24:07.784062037 +0000 UTC m=+51.590473254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dsmtv" (UID: "d908e88c-afcd-4b67-97ef-b36595506a36") : secret "networking-console-plugin-cert" not found
Jul 23 00:24:06.024291 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:06.024248    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs" event={"ID":"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e","Type":"ContainerStarted","Data":"449280d3c2f6d6887a14acaf8607b75ac7d9678becc8da24f0143382da6c61a4"}
Jul 23 00:24:06.024803 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:06.024311    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs" event={"ID":"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e","Type":"ContainerStarted","Data":"80a01c391b60f4ba171a2201b5921523a43ff8afabdb8b60f854e8130d463662"}
Jul 23 00:24:06.024803 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:06.024327    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs" event={"ID":"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e","Type":"ContainerStarted","Data":"1ca2547082229cceb1d04027ba49321775bdef4e65e637c140368e69ab595cb2"}
Jul 23 00:24:06.025684 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:06.025657    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65" event={"ID":"cd2d8df2-d752-4524-991f-aa1def67e57e","Type":"ContainerStarted","Data":"2ea645fa79179ee9da00281c84bf1a89abe0ab81ea758ed2e0e7e5abd3a466b6"}
Jul 23 00:24:06.025785 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:06.025711    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:24:06.027192 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:06.027168    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k" event={"ID":"2d9c04b3-b899-4884-9e89-0037f29f8a0f","Type":"ContainerStarted","Data":"1ef61da5d854dd5e7402ec0b20bdd47de821dfaefb3d7ca51645dd885271ade4"}
Jul 23 00:24:06.027698 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:06.027680    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:24:06.044353 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:06.044313    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs" podStartSLOduration=1.282046904 podStartE2EDuration="8.044300709s" totalImagesPullingTime="6.762253805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:24:06.043079356 +0000 UTC m=+49.849490589" watchObservedRunningTime="2026-07-23 00:24:06.044300709 +0000 UTC m=+49.850711920"
Jul 23 00:24:06.059356 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:06.059276    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k" podStartSLOduration=1.263168144 podStartE2EDuration="8.059259903s" totalImagesPullingTime="6.796091759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:24:06.058832879 +0000 UTC m=+49.865244110" watchObservedRunningTime="2026-07-23 00:24:06.059259903 +0000 UTC m=+49.865671135"
Jul 23 00:24:06.074790 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:06.074745    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65" podStartSLOduration=1.302659907 podStartE2EDuration="8.074732015s" totalImagesPullingTime="6.772072108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:24:06.074527866 +0000 UTC m=+49.880939099" watchObservedRunningTime="2026-07-23 00:24:06.074732015 +0000 UTC m=+49.881143247"
Jul 23 00:24:07.857892 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:07.857853    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:24:07.857892 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:07.857892    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:24:07.858415 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:07.857985    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:24:07.858415 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:07.857990    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 00:24:07.858415 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:07.858036    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls podName:4fbb34aa-fae2-48b7-bda5-57e0031af622 nodeName:}" failed. No retries permitted until 2026-07-23 00:24:23.858023242 +0000 UTC m=+67.664434453 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls") pod "dns-default-jgpkp" (UID: "4fbb34aa-fae2-48b7-bda5-57e0031af622") : secret "dns-default-metrics-tls" not found
Jul 23 00:24:07.858415 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:07.858034    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:24:07.858415 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:07.858049    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert podName:d908e88c-afcd-4b67-97ef-b36595506a36 nodeName:}" failed. No retries permitted until 2026-07-23 00:24:23.858043867 +0000 UTC m=+67.664455078 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dsmtv" (UID: "d908e88c-afcd-4b67-97ef-b36595506a36") : secret "networking-console-plugin-cert" not found
Jul 23 00:24:07.858415 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:07.858142    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:24:07.858415 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:07.858159    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868df8cc95-gjff5: secret "image-registry-tls" not found
Jul 23 00:24:07.858415 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:07.858209    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls podName:6b5feda6-e797-4219-9e8a-bc0f0a26909f nodeName:}" failed. No retries permitted until 2026-07-23 00:24:23.858196219 +0000 UTC m=+67.664607430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls") pod "image-registry-868df8cc95-gjff5" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f") : secret "image-registry-tls" not found
Jul 23 00:24:16.960303 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:16.960275    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g9g57"
Jul 23 00:24:22.477874 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.477824    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:24:22.477874 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.477875    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:24:22.478313 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.477938    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9646\" (UniqueName: \"kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646\") pod \"network-check-target-nzvr2\" (UID: \"c5eeb225-2f11-4dd2-b57f-933946195251\") " pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:24:22.481908 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.481881    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 00:24:22.481908 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.481885    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 00:24:22.482079 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.481888    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 00:24:22.488093 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:22.488075    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 23 00:24:22.488180 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:22.488139    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs podName:7c65b730-bf89-4090-b8a7-cad9a31bf810 nodeName:}" failed. No retries permitted until 2026-07-23 00:25:26.488124596 +0000 UTC m=+130.294535807 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs") pod "network-metrics-daemon-6dn2c" (UID: "7c65b730-bf89-4090-b8a7-cad9a31bf810") : secret "metrics-daemon-secret" not found
Jul 23 00:24:22.490227 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.490212    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/64edf751-2ca9-432a-a626-2c23a5f79032-original-pull-secret\") pod \"global-pull-secret-syncer-l8zzv\" (UID: \"64edf751-2ca9-432a-a626-2c23a5f79032\") " pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:24:22.490582 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.490568    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 00:24:22.500974 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.500947    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9646\" (UniqueName: \"kubernetes.io/projected/c5eeb225-2f11-4dd2-b57f-933946195251-kube-api-access-v9646\") pod \"network-check-target-nzvr2\" (UID: \"c5eeb225-2f11-4dd2-b57f-933946195251\") " pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:24:22.601753 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.601725    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-l8zzv"
Jul 23 00:24:22.611317 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.611299    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c75lg\""
Jul 23 00:24:22.619223 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.619204    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:24:22.733679 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.733605    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-l8zzv"]
Jul 23 00:24:22.736277 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:24:22.736240    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64edf751_2ca9_432a_a626_2c23a5f79032.slice/crio-cacf53d794c39e605023b3e46ff01c740946a0035c91a0e8349c6b2ad1709db8 WatchSource:0}: Error finding container cacf53d794c39e605023b3e46ff01c740946a0035c91a0e8349c6b2ad1709db8: Status 404 returned error can't find the container with id cacf53d794c39e605023b3e46ff01c740946a0035c91a0e8349c6b2ad1709db8
Jul 23 00:24:22.748341 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:22.748320    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nzvr2"]
Jul 23 00:24:22.750420 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:24:22.750390    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5eeb225_2f11_4dd2_b57f_933946195251.slice/crio-df2bb8f38d273f7cbd49af9c46dad56cd3c995e89cc52c148f79a39e54941a5d WatchSource:0}: Error finding container df2bb8f38d273f7cbd49af9c46dad56cd3c995e89cc52c148f79a39e54941a5d: Status 404 returned error can't find the container with id df2bb8f38d273f7cbd49af9c46dad56cd3c995e89cc52c148f79a39e54941a5d
Jul 23 00:24:23.075107 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:23.075026    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l8zzv" event={"ID":"64edf751-2ca9-432a-a626-2c23a5f79032","Type":"ContainerStarted","Data":"cacf53d794c39e605023b3e46ff01c740946a0035c91a0e8349c6b2ad1709db8"}
Jul 23 00:24:23.076022 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:23.076002    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nzvr2" event={"ID":"c5eeb225-2f11-4dd2-b57f-933946195251","Type":"ContainerStarted","Data":"df2bb8f38d273f7cbd49af9c46dad56cd3c995e89cc52c148f79a39e54941a5d"}
Jul 23 00:24:23.889507 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:23.889473    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:24:23.889986 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:23.889581    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:24:23.889986 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:23.889613    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:24:23.889986 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:23.889618    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:24:23.889986 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:23.889641    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868df8cc95-gjff5: secret "image-registry-tls" not found
Jul 23 00:24:23.889986 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:23.889713    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls podName:6b5feda6-e797-4219-9e8a-bc0f0a26909f nodeName:}" failed. No retries permitted until 2026-07-23 00:24:55.889691137 +0000 UTC m=+99.696102362 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls") pod "image-registry-868df8cc95-gjff5" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f") : secret "image-registry-tls" not found
Jul 23 00:24:23.889986 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:23.889733    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:24:23.889986 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:23.889741    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 00:24:23.889986 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:23.889792    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls podName:4fbb34aa-fae2-48b7-bda5-57e0031af622 nodeName:}" failed. No retries permitted until 2026-07-23 00:24:55.889775365 +0000 UTC m=+99.696186577 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls") pod "dns-default-jgpkp" (UID: "4fbb34aa-fae2-48b7-bda5-57e0031af622") : secret "dns-default-metrics-tls" not found
Jul 23 00:24:23.889986 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:23.889812    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert podName:d908e88c-afcd-4b67-97ef-b36595506a36 nodeName:}" failed. No retries permitted until 2026-07-23 00:24:55.889802211 +0000 UTC m=+99.696213426 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dsmtv" (UID: "d908e88c-afcd-4b67-97ef-b36595506a36") : secret "networking-console-plugin-cert" not found
Jul 23 00:24:27.095325 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:27.095285    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nzvr2" event={"ID":"c5eeb225-2f11-4dd2-b57f-933946195251","Type":"ContainerStarted","Data":"452b30e95a9fd61f8f5f602e3c2e743c7a5fad58129dbc55e3339127bd181a6d"}
Jul 23 00:24:27.095809 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:27.095421    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:24:27.112158 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:27.112105    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nzvr2" podStartSLOduration=66.702436197 podStartE2EDuration="1m10.112085648s" totalImagesPullingTime="3.409649451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:24:27.112050315 +0000 UTC m=+70.918461551" watchObservedRunningTime="2026-07-23 00:24:27.112085648 +0000 UTC m=+70.918496882"
Jul 23 00:24:28.099807 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:28.099767    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-l8zzv" event={"ID":"64edf751-2ca9-432a-a626-2c23a5f79032","Type":"ContainerStarted","Data":"81a65a6d0642b20003c86073f8881478a1c12e25835f2551d8fce6ded0ff7016"}
Jul 23 00:24:28.116657 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:28.116611    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-l8zzv" podStartSLOduration=65.965710446 podStartE2EDuration="1m11.116596796s" totalImagesPullingTime="5.15088635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:24:28.115907744 +0000 UTC m=+71.922318980" watchObservedRunningTime="2026-07-23 00:24:28.116596796 +0000 UTC m=+71.923008074"
Jul 23 00:24:55.935936 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:55.935881    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:24:55.935936 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:55.935936    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:24:55.936516 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:55.935970    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:24:55.936516 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:55.936049    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 00:24:55.936516 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:55.936094    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:24:55.936516 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:55.936127    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:24:55.936516 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:55.936142    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868df8cc95-gjff5: secret "image-registry-tls" not found
Jul 23 00:24:55.936516 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:55.936144    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert podName:d908e88c-afcd-4b67-97ef-b36595506a36 nodeName:}" failed. No retries permitted until 2026-07-23 00:25:59.936124225 +0000 UTC m=+163.742535456 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dsmtv" (UID: "d908e88c-afcd-4b67-97ef-b36595506a36") : secret "networking-console-plugin-cert" not found
Jul 23 00:24:55.936516 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:55.936212    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls podName:4fbb34aa-fae2-48b7-bda5-57e0031af622 nodeName:}" failed. No retries permitted until 2026-07-23 00:25:59.936196181 +0000 UTC m=+163.742607398 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls") pod "dns-default-jgpkp" (UID: "4fbb34aa-fae2-48b7-bda5-57e0031af622") : secret "dns-default-metrics-tls" not found
Jul 23 00:24:55.936516 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:24:55.936226    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls podName:6b5feda6-e797-4219-9e8a-bc0f0a26909f nodeName:}" failed. No retries permitted until 2026-07-23 00:25:59.936218411 +0000 UTC m=+163.742629622 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls") pod "image-registry-868df8cc95-gjff5" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f") : secret "image-registry-tls" not found
Jul 23 00:24:58.102571 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:24:58.102544    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nzvr2"
Jul 23 00:25:26.562033 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:25:26.561977    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:25:26.562579 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:25:26.562129    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 23 00:25:26.562579 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:25:26.562199    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs podName:7c65b730-bf89-4090-b8a7-cad9a31bf810 nodeName:}" failed. No retries permitted until 2026-07-23 00:27:28.562181878 +0000 UTC m=+252.368593090 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs") pod "network-metrics-daemon-6dn2c" (UID: "7c65b730-bf89-4090-b8a7-cad9a31bf810") : secret "metrics-daemon-secret" not found
Jul 23 00:25:54.969858 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:25:54.969812    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv" podUID="d908e88c-afcd-4b67-97ef-b36595506a36"
Jul 23 00:25:54.972967 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:25:54.972941    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-868df8cc95-gjff5" podUID="6b5feda6-e797-4219-9e8a-bc0f0a26909f"
Jul 23 00:25:54.988287 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:25:54.988261    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jgpkp" podUID="4fbb34aa-fae2-48b7-bda5-57e0031af622"
Jul 23 00:25:55.345813 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:25:55.345725    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:25:55.345813 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:25:55.345758    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:25:55.346015 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:25:55.345874    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:25:56.794322 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:25:56.794287    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:26:00.012870 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:00.012812    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:26:00.012870 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:00.012867    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:26:00.013305 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:26:00.012976    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 00:26:00.013305 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:26:00.013012    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:26:00.013305 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:26:00.013045    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert podName:d908e88c-afcd-4b67-97ef-b36595506a36 nodeName:}" failed. No retries permitted until 2026-07-23 00:28:02.013027775 +0000 UTC m=+285.819438989 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dsmtv" (UID: "d908e88c-afcd-4b67-97ef-b36595506a36") : secret "networking-console-plugin-cert" not found
Jul 23 00:26:00.013305 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:00.013068    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:26:00.013305 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:26:00.013161    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls podName:4fbb34aa-fae2-48b7-bda5-57e0031af622 nodeName:}" failed. No retries permitted until 2026-07-23 00:28:02.013154147 +0000 UTC m=+285.819565358 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls") pod "dns-default-jgpkp" (UID: "4fbb34aa-fae2-48b7-bda5-57e0031af622") : secret "dns-default-metrics-tls" not found
Jul 23 00:26:00.013305 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:26:00.013188    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:26:00.013305 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:26:00.013210    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868df8cc95-gjff5: secret "image-registry-tls" not found
Jul 23 00:26:00.013305 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:26:00.013273    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls podName:6b5feda6-e797-4219-9e8a-bc0f0a26909f nodeName:}" failed. No retries permitted until 2026-07-23 00:28:02.013256137 +0000 UTC m=+285.819667349 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls") pod "image-registry-868df8cc95-gjff5" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f") : secret "image-registry-tls" not found
Jul 23 00:26:06.027087 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:06.026991    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65" podUID="cd2d8df2-d752-4524-991f-aa1def67e57e" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.11:8000/readyz\": dial tcp 10.132.0.11:8000: connect: connection refused"
Jul 23 00:26:06.380166 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:06.380083    2561 generic.go:356] "Generic (PLEG): container finished" podID="2d9c04b3-b899-4884-9e89-0037f29f8a0f" containerID="1ef61da5d854dd5e7402ec0b20bdd47de821dfaefb3d7ca51645dd885271ade4" exitCode=255
Jul 23 00:26:06.380166 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:06.380155    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k" event={"ID":"2d9c04b3-b899-4884-9e89-0037f29f8a0f","Type":"ContainerDied","Data":"1ef61da5d854dd5e7402ec0b20bdd47de821dfaefb3d7ca51645dd885271ade4"}
Jul 23 00:26:06.380544 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:06.380522    2561 scope.go:117] "RemoveContainer" containerID="1ef61da5d854dd5e7402ec0b20bdd47de821dfaefb3d7ca51645dd885271ade4"
Jul 23 00:26:06.381823 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:06.381801    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd2d8df2-d752-4524-991f-aa1def67e57e" containerID="2ea645fa79179ee9da00281c84bf1a89abe0ab81ea758ed2e0e7e5abd3a466b6" exitCode=1
Jul 23 00:26:06.381879 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:06.381842    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65" event={"ID":"cd2d8df2-d752-4524-991f-aa1def67e57e","Type":"ContainerDied","Data":"2ea645fa79179ee9da00281c84bf1a89abe0ab81ea758ed2e0e7e5abd3a466b6"}
Jul 23 00:26:06.382157 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:06.382143    2561 scope.go:117] "RemoveContainer" containerID="2ea645fa79179ee9da00281c84bf1a89abe0ab81ea758ed2e0e7e5abd3a466b6"
Jul 23 00:26:07.386101 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:07.386060    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65" event={"ID":"cd2d8df2-d752-4524-991f-aa1def67e57e","Type":"ContainerStarted","Data":"3c663ee834b0c99e0cc7f58094df64e3a0ebbcd217d0ac1d2ff1a4cb0fdc67f1"}
Jul 23 00:26:07.386582 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:07.386346    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:26:07.387021 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:07.387000    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:26:07.387926 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:07.387904    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k" event={"ID":"2d9c04b3-b899-4884-9e89-0037f29f8a0f","Type":"ContainerStarted","Data":"cdd56cc3f5eecc3d803db440755599c8a89d144c15091f825ed153e7b68df69c"}
Jul 23 00:26:11.781895 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:26:11.781859    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:27:25.590454 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.590419    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hc78w"]
Jul 23 00:27:25.592572 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.592553    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.595128 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.595107    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 00:27:25.595223 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.595131    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fm2gr\""
Jul 23 00:27:25.596282 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.596264    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 00:27:25.596414 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.596337    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 00:27:25.596600 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.596584    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 00:27:25.604879 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.604859    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hc78w"]
Jul 23 00:27:25.646822 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.646788    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f0d486fc-9ffd-46f0-bbc4-044247168063-crio-socket\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.646999 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.646859    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0d486fc-9ffd-46f0-bbc4-044247168063-data-volume\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.646999 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.646889    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.646999 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.646936    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmbdh\" (UniqueName: \"kubernetes.io/projected/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-api-access-kmbdh\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.646999 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.646957    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.747951 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.747923    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmbdh\" (UniqueName: \"kubernetes.io/projected/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-api-access-kmbdh\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.748102 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.747956    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.748102 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.747979    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f0d486fc-9ffd-46f0-bbc4-044247168063-crio-socket\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.748102 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.748047    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0d486fc-9ffd-46f0-bbc4-044247168063-data-volume\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.748102 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.748074    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.748102 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:25.748076    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 00:27:25.748331 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.748135    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f0d486fc-9ffd-46f0-bbc4-044247168063-crio-socket\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.748331 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:25.748145    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 00:27:25.748331 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:25.748160    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls podName:f0d486fc-9ffd-46f0-bbc4-044247168063 nodeName:}" failed. No retries permitted until 2026-07-23 00:27:26.248138925 +0000 UTC m=+250.054550151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hc78w" (UID: "f0d486fc-9ffd-46f0-bbc4-044247168063") : secret "insights-runtime-extractor-tls" not found
Jul 23 00:27:25.748331 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:25.748225    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm podName:f0d486fc-9ffd-46f0-bbc4-044247168063 nodeName:}" failed. No retries permitted until 2026-07-23 00:27:26.248208866 +0000 UTC m=+250.054620089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hc78w" (UID: "f0d486fc-9ffd-46f0-bbc4-044247168063") : configmap "kube-rbac-proxy" not found
Jul 23 00:27:25.748499 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.748416    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f0d486fc-9ffd-46f0-bbc4-044247168063-data-volume\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:25.756460 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:25.756438    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmbdh\" (UniqueName: \"kubernetes.io/projected/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-api-access-kmbdh\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:26.251475 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:26.251435    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:26.251680 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:26.251500    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:26.251680 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:26.251558    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 00:27:26.251680 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:26.251595    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 00:27:26.251680 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:26.251632    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm podName:f0d486fc-9ffd-46f0-bbc4-044247168063 nodeName:}" failed. No retries permitted until 2026-07-23 00:27:27.251613512 +0000 UTC m=+251.058024728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hc78w" (UID: "f0d486fc-9ffd-46f0-bbc4-044247168063") : configmap "kube-rbac-proxy" not found
Jul 23 00:27:26.251680 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:26.251649    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls podName:f0d486fc-9ffd-46f0-bbc4-044247168063 nodeName:}" failed. No retries permitted until 2026-07-23 00:27:27.251641667 +0000 UTC m=+251.058052885 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hc78w" (UID: "f0d486fc-9ffd-46f0-bbc4-044247168063") : secret "insights-runtime-extractor-tls" not found
Jul 23 00:27:27.256135 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:27.256098    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:27.256633 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:27.256198    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:27.256633 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:27.256223    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 00:27:27.256633 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:27.256306    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm podName:f0d486fc-9ffd-46f0-bbc4-044247168063 nodeName:}" failed. No retries permitted until 2026-07-23 00:27:29.256282522 +0000 UTC m=+253.062693735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hc78w" (UID: "f0d486fc-9ffd-46f0-bbc4-044247168063") : configmap "kube-rbac-proxy" not found
Jul 23 00:27:27.256633 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:27.256319    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 00:27:27.256633 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:27.256397    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls podName:f0d486fc-9ffd-46f0-bbc4-044247168063 nodeName:}" failed. No retries permitted until 2026-07-23 00:27:29.256382354 +0000 UTC m=+253.062793582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hc78w" (UID: "f0d486fc-9ffd-46f0-bbc4-044247168063") : secret "insights-runtime-extractor-tls" not found
Jul 23 00:27:27.468185 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:27.468158    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5z66q_4fdfd2d4-f612-4ee0-b4cd-66417c138705/dns-node-resolver/0.log"
Jul 23 00:27:28.565666 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:28.565616    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:27:28.566083 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:28.565759    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 23 00:27:28.566083 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:28.565840    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs podName:7c65b730-bf89-4090-b8a7-cad9a31bf810 nodeName:}" failed. No retries permitted until 2026-07-23 00:29:30.565824209 +0000 UTC m=+374.372235423 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs") pod "network-metrics-daemon-6dn2c" (UID: "7c65b730-bf89-4090-b8a7-cad9a31bf810") : secret "metrics-daemon-secret" not found
Jul 23 00:27:28.665036 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:28.665009    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hsf6v_410e2099-4588-45d5-a93c-cb4066032492/node-ca/0.log"
Jul 23 00:27:29.271323 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:29.271283    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:29.271535 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:29.271349    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:29.271535 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:29.271441    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 00:27:29.271535 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:29.271469    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 00:27:29.271535 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:29.271523    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls podName:f0d486fc-9ffd-46f0-bbc4-044247168063 nodeName:}" failed. No retries permitted until 2026-07-23 00:27:33.271509274 +0000 UTC m=+257.077920485 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hc78w" (UID: "f0d486fc-9ffd-46f0-bbc4-044247168063") : secret "insights-runtime-extractor-tls" not found
Jul 23 00:27:29.271535 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:29.271536    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm podName:f0d486fc-9ffd-46f0-bbc4-044247168063 nodeName:}" failed. No retries permitted until 2026-07-23 00:27:33.271529591 +0000 UTC m=+257.077940802 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hc78w" (UID: "f0d486fc-9ffd-46f0-bbc4-044247168063") : configmap "kube-rbac-proxy" not found
Jul 23 00:27:29.664477 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:29.664399    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xn4kv_197306dc-60be-49d8-854c-dba425eeb32c/serve-healthcheck-canary/0.log"
Jul 23 00:27:33.301774 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:33.301733    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:33.302259 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:33.301875    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:33.302259 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:33.301890    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 00:27:33.302259 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:33.301957    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls podName:f0d486fc-9ffd-46f0-bbc4-044247168063 nodeName:}" failed. No retries permitted until 2026-07-23 00:27:41.301938517 +0000 UTC m=+265.108349731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hc78w" (UID: "f0d486fc-9ffd-46f0-bbc4-044247168063") : secret "insights-runtime-extractor-tls" not found
Jul 23 00:27:33.302259 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:33.301961    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 00:27:33.302259 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:33.302010    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm podName:f0d486fc-9ffd-46f0-bbc4-044247168063 nodeName:}" failed. No retries permitted until 2026-07-23 00:27:41.301999276 +0000 UTC m=+265.108410487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hc78w" (UID: "f0d486fc-9ffd-46f0-bbc4-044247168063") : configmap "kube-rbac-proxy" not found
Jul 23 00:27:41.362119 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:41.362025    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:41.362717 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:41.362194    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:41.362717 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:41.362198    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 00:27:41.362717 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:41.362312    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls podName:f0d486fc-9ffd-46f0-bbc4-044247168063 nodeName:}" failed. No retries permitted until 2026-07-23 00:27:57.362297225 +0000 UTC m=+281.168708436 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hc78w" (UID: "f0d486fc-9ffd-46f0-bbc4-044247168063") : secret "insights-runtime-extractor-tls" not found
Jul 23 00:27:41.362891 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:41.362778    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f0d486fc-9ffd-46f0-bbc4-044247168063-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:57.388807 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:57.388774    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:57.390953 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:57.390936    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f0d486fc-9ffd-46f0-bbc4-044247168063-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hc78w\" (UID: \"f0d486fc-9ffd-46f0-bbc4-044247168063\") " pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:57.400924 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:57.400903    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hc78w"
Jul 23 00:27:57.523008 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:57.522978    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hc78w"]
Jul 23 00:27:57.525922 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:27:57.525895    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d486fc_9ffd_46f0_bbc4_044247168063.slice/crio-b334dc3196d6a2c7e5f4072e5d4d694a3e31e4f96b8e86662e1e1fd98799aaca WatchSource:0}: Error finding container b334dc3196d6a2c7e5f4072e5d4d694a3e31e4f96b8e86662e1e1fd98799aaca: Status 404 returned error can't find the container with id b334dc3196d6a2c7e5f4072e5d4d694a3e31e4f96b8e86662e1e1fd98799aaca
Jul 23 00:27:57.709753 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:57.709718    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hc78w" event={"ID":"f0d486fc-9ffd-46f0-bbc4-044247168063","Type":"ContainerStarted","Data":"b334dc3196d6a2c7e5f4072e5d4d694a3e31e4f96b8e86662e1e1fd98799aaca"}
Jul 23 00:27:58.346987 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:58.346936    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv" podUID="d908e88c-afcd-4b67-97ef-b36595506a36"
Jul 23 00:27:58.346987 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:58.346937    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-868df8cc95-gjff5" podUID="6b5feda6-e797-4219-9e8a-bc0f0a26909f"
Jul 23 00:27:58.347245 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:27:58.346936    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jgpkp" podUID="4fbb34aa-fae2-48b7-bda5-57e0031af622"
Jul 23 00:27:58.713170 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:58.713140    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:27:58.713612 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:58.713140    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:27:58.713612 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:27:58.713143    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:28:02.029424 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.029382    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:28:02.029866 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.029434    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:28:02.029866 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.029483    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:28:02.032001 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.031978    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d908e88c-afcd-4b67-97ef-b36595506a36-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dsmtv\" (UID: \"d908e88c-afcd-4b67-97ef-b36595506a36\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:28:02.032138 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.031980    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4fbb34aa-fae2-48b7-bda5-57e0031af622-metrics-tls\") pod \"dns-default-jgpkp\" (UID: \"4fbb34aa-fae2-48b7-bda5-57e0031af622\") " pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:28:02.032192 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.032136    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"image-registry-868df8cc95-gjff5\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") " pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:28:02.318273 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.318202    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fhdn8\""
Jul 23 00:28:02.318273 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.318236    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6v7ht\""
Jul 23 00:28:02.318505 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.318311    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tvzb2\""
Jul 23 00:28:02.325398 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.325373    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"
Jul 23 00:28:02.325525 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.325404    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:28:02.325525 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.325467    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:28:02.476269 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.476239    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jgpkp"]
Jul 23 00:28:02.479796 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:28:02.479762    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fbb34aa_fae2_48b7_bda5_57e0031af622.slice/crio-0b2529a5208f248cb1ef9d264f459ca55ab14050cb76999c462a7b6ecb3aab7c WatchSource:0}: Error finding container 0b2529a5208f248cb1ef9d264f459ca55ab14050cb76999c462a7b6ecb3aab7c: Status 404 returned error can't find the container with id 0b2529a5208f248cb1ef9d264f459ca55ab14050cb76999c462a7b6ecb3aab7c
Jul 23 00:28:02.705306 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.705078    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv"]
Jul 23 00:28:02.707621 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:28:02.707592    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd908e88c_afcd_4b67_97ef_b36595506a36.slice/crio-af23ff1c1e0a02461932ac1f8228adf086c90b618bc18c8d5d3e338078061464 WatchSource:0}: Error finding container af23ff1c1e0a02461932ac1f8228adf086c90b618bc18c8d5d3e338078061464: Status 404 returned error can't find the container with id af23ff1c1e0a02461932ac1f8228adf086c90b618bc18c8d5d3e338078061464
Jul 23 00:28:02.710788 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.710769    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-868df8cc95-gjff5"]
Jul 23 00:28:02.713056 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:28:02.713032    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b5feda6_e797_4219_9e8a_bc0f0a26909f.slice/crio-59ce9f1a1935a3d0923281fee1bafba1f38dc7c8888221656ab118bc891b2afe WatchSource:0}: Error finding container 59ce9f1a1935a3d0923281fee1bafba1f38dc7c8888221656ab118bc891b2afe: Status 404 returned error can't find the container with id 59ce9f1a1935a3d0923281fee1bafba1f38dc7c8888221656ab118bc891b2afe
Jul 23 00:28:02.727721 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.727696    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hc78w" event={"ID":"f0d486fc-9ffd-46f0-bbc4-044247168063","Type":"ContainerStarted","Data":"5f1c0e880d151c762a6301af5e8ce9ca3cd7f45f643fd5275f7b00fd474788ba"}
Jul 23 00:28:02.728924 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.728901    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv" event={"ID":"d908e88c-afcd-4b67-97ef-b36595506a36","Type":"ContainerStarted","Data":"af23ff1c1e0a02461932ac1f8228adf086c90b618bc18c8d5d3e338078061464"}
Jul 23 00:28:02.730134 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.730114    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868df8cc95-gjff5" event={"ID":"6b5feda6-e797-4219-9e8a-bc0f0a26909f","Type":"ContainerStarted","Data":"59ce9f1a1935a3d0923281fee1bafba1f38dc7c8888221656ab118bc891b2afe"}
Jul 23 00:28:02.731124 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:02.731105    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jgpkp" event={"ID":"4fbb34aa-fae2-48b7-bda5-57e0031af622","Type":"ContainerStarted","Data":"0b2529a5208f248cb1ef9d264f459ca55ab14050cb76999c462a7b6ecb3aab7c"}
Jul 23 00:28:03.738493 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:03.738455    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868df8cc95-gjff5" event={"ID":"6b5feda6-e797-4219-9e8a-bc0f0a26909f","Type":"ContainerStarted","Data":"abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720"}
Jul 23 00:28:03.739305 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:03.739283    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:28:03.764202 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:03.764145    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-868df8cc95-gjff5" podStartSLOduration=286.76412526 podStartE2EDuration="4m46.76412526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:28:03.762142412 +0000 UTC m=+287.568553659" watchObservedRunningTime="2026-07-23 00:28:03.76412526 +0000 UTC m=+287.570536495"
Jul 23 00:28:06.752224 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.752189    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hc78w" event={"ID":"f0d486fc-9ffd-46f0-bbc4-044247168063","Type":"ContainerStarted","Data":"db4c30280b8ba990598f88ad5419be93906536a786a6af488acb6d87d1fb3422"}
Jul 23 00:28:06.753674 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.753646    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv" event={"ID":"d908e88c-afcd-4b67-97ef-b36595506a36","Type":"ContainerStarted","Data":"928598b77ad6d6f29367cb352359a8f480c4361e206ec79c4084c242129a036c"}
Jul 23 00:28:06.755325 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.755302    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd2d8df2-d752-4524-991f-aa1def67e57e" containerID="3c663ee834b0c99e0cc7f58094df64e3a0ebbcd217d0ac1d2ff1a4cb0fdc67f1" exitCode=1
Jul 23 00:28:06.755464 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.755390    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65" event={"ID":"cd2d8df2-d752-4524-991f-aa1def67e57e","Type":"ContainerDied","Data":"3c663ee834b0c99e0cc7f58094df64e3a0ebbcd217d0ac1d2ff1a4cb0fdc67f1"}
Jul 23 00:28:06.755464 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.755446    2561 scope.go:117] "RemoveContainer" containerID="2ea645fa79179ee9da00281c84bf1a89abe0ab81ea758ed2e0e7e5abd3a466b6"
Jul 23 00:28:06.755735 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.755717    2561 scope.go:117] "RemoveContainer" containerID="3c663ee834b0c99e0cc7f58094df64e3a0ebbcd217d0ac1d2ff1a4cb0fdc67f1"
Jul 23 00:28:06.755889 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:28:06.755869    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-7c5ddd655b-ksf65_open-cluster-management-agent-addon(cd2d8df2-d752-4524-991f-aa1def67e57e)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65" podUID="cd2d8df2-d752-4524-991f-aa1def67e57e"
Jul 23 00:28:06.757119 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.757093    2561 generic.go:356] "Generic (PLEG): container finished" podID="2d9c04b3-b899-4884-9e89-0037f29f8a0f" containerID="cdd56cc3f5eecc3d803db440755599c8a89d144c15091f825ed153e7b68df69c" exitCode=255
Jul 23 00:28:06.757226 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.757182    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k" event={"ID":"2d9c04b3-b899-4884-9e89-0037f29f8a0f","Type":"ContainerDied","Data":"cdd56cc3f5eecc3d803db440755599c8a89d144c15091f825ed153e7b68df69c"}
Jul 23 00:28:06.757490 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.757473    2561 scope.go:117] "RemoveContainer" containerID="cdd56cc3f5eecc3d803db440755599c8a89d144c15091f825ed153e7b68df69c"
Jul 23 00:28:06.757657 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:28:06.757636    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-6cb57d4b66-j974k_open-cluster-management-agent-addon(2d9c04b3-b899-4884-9e89-0037f29f8a0f)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k" podUID="2d9c04b3-b899-4884-9e89-0037f29f8a0f"
Jul 23 00:28:06.759164 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.759142    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jgpkp" event={"ID":"4fbb34aa-fae2-48b7-bda5-57e0031af622","Type":"ContainerStarted","Data":"ebfe7e3d661135032adaf9a85487a4e47c9e7e67812249b034d4ca4a65a17ed7"}
Jul 23 00:28:06.759273 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.759182    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jgpkp" event={"ID":"4fbb34aa-fae2-48b7-bda5-57e0031af622","Type":"ContainerStarted","Data":"77960bf86b2383adb6cbffa032950d52a49ab679229d258b21a731e5e4d0b234"}
Jul 23 00:28:06.759273 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.759212    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:28:06.767135 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.767115    2561 scope.go:117] "RemoveContainer" containerID="1ef61da5d854dd5e7402ec0b20bdd47de821dfaefb3d7ca51645dd885271ade4"
Jul 23 00:28:06.770897 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.770849    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dsmtv" podStartSLOduration=279.057179935 podStartE2EDuration="4m42.770835663s" totalImagesPullingTime="3.713655728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:28:06.770350819 +0000 UTC m=+290.576762052" watchObservedRunningTime="2026-07-23 00:28:06.770835663 +0000 UTC m=+290.577246903"
Jul 23 00:28:06.824547 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:06.824495    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jgpkp" podStartSLOduration=251.878238159 podStartE2EDuration="4m15.824479218s" totalImagesPullingTime="3.946241059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:28:06.823355381 +0000 UTC m=+290.629766627" watchObservedRunningTime="2026-07-23 00:28:06.824479218 +0000 UTC m=+290.630890450"
Jul 23 00:28:07.386960 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:07.386926    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:28:07.765078 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:07.765050    2561 scope.go:117] "RemoveContainer" containerID="3c663ee834b0c99e0cc7f58094df64e3a0ebbcd217d0ac1d2ff1a4cb0fdc67f1"
Jul 23 00:28:07.765569 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:28:07.765221    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-7c5ddd655b-ksf65_open-cluster-management-agent-addon(cd2d8df2-d752-4524-991f-aa1def67e57e)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65" podUID="cd2d8df2-d752-4524-991f-aa1def67e57e"
Jul 23 00:28:08.587065 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.587032    2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k"
Jul 23 00:28:08.587426 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.587411    2561 scope.go:117] "RemoveContainer" containerID="cdd56cc3f5eecc3d803db440755599c8a89d144c15091f825ed153e7b68df69c"
Jul 23 00:28:08.587621 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:28:08.587597    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-6cb57d4b66-j974k_open-cluster-management-agent-addon(2d9c04b3-b899-4884-9e89-0037f29f8a0f)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k" podUID="2d9c04b3-b899-4884-9e89-0037f29f8a0f"
Jul 23 00:28:08.594674 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.594655    2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:28:08.773047 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.773020    2561 scope.go:117] "RemoveContainer" containerID="3c663ee834b0c99e0cc7f58094df64e3a0ebbcd217d0ac1d2ff1a4cb0fdc67f1"
Jul 23 00:28:08.773501 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:28:08.773190    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-7c5ddd655b-ksf65_open-cluster-management-agent-addon(cd2d8df2-d752-4524-991f-aa1def67e57e)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65" podUID="cd2d8df2-d752-4524-991f-aa1def67e57e"
Jul 23 00:28:08.847569 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.847473    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wf4m9"]
Jul 23 00:28:08.856590 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.856565    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:08.859225 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.859199    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kwh25\""
Jul 23 00:28:08.859353 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.859201    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 00:28:08.859713 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.859693    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 00:28:08.859815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.859760    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 00:28:08.859815 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.859799    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 00:28:08.860052 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.860034    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 00:28:08.860202 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.860190    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 00:28:08.990560 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.990521    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-wtmp\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:08.990741 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.990579    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-sys\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:08.990741 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.990661    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-textfile\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:08.990741 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.990706    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-metrics-client-ca\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:08.990910 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.990765    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnvxr\" (UniqueName: \"kubernetes.io/projected/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-kube-api-access-lnvxr\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:08.990910 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.990798    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-tls\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:08.990910 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.990823    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:08.990910 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.990869    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-root\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:08.991111 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:08.990922    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-accelerators-collector-config\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092127 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092090    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-wtmp\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092333 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092158    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-sys\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092333 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092193    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-textfile\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092333 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092220    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-metrics-client-ca\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092333 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092247    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnvxr\" (UniqueName: \"kubernetes.io/projected/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-kube-api-access-lnvxr\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092333 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092280    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-tls\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092333 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092285    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-wtmp\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092333 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092315    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092772 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092388    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-root\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092772 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092427    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-accelerators-collector-config\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092772 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092286    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-sys\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092772 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092598    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-textfile\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092772 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092632    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-root\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.092944 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.092875    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-metrics-client-ca\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.093098 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.093073    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-accelerators-collector-config\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.095148 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.095127    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-tls\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.095571 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.095550    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.100201 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.100150    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnvxr\" (UniqueName: \"kubernetes.io/projected/a7e819d6-e8d1-47ec-acfc-d58fb660dc66-kube-api-access-lnvxr\") pod \"node-exporter-wf4m9\" (UID: \"a7e819d6-e8d1-47ec-acfc-d58fb660dc66\") " pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.167467 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:09.167434    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wf4m9"
Jul 23 00:28:09.938486 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:28:09.938455    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e819d6_e8d1_47ec_acfc_d58fb660dc66.slice/crio-69e3505eeb2781018a7b2745ce8354548bb9840424eae64b9c3c06d10bd119cb WatchSource:0}: Error finding container 69e3505eeb2781018a7b2745ce8354548bb9840424eae64b9c3c06d10bd119cb: Status 404 returned error can't find the container with id 69e3505eeb2781018a7b2745ce8354548bb9840424eae64b9c3c06d10bd119cb
Jul 23 00:28:10.785732 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:10.785702    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wf4m9" event={"ID":"a7e819d6-e8d1-47ec-acfc-d58fb660dc66","Type":"ContainerStarted","Data":"69e3505eeb2781018a7b2745ce8354548bb9840424eae64b9c3c06d10bd119cb"}
Jul 23 00:28:10.785833 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:10.785765    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hc78w" event={"ID":"f0d486fc-9ffd-46f0-bbc4-044247168063","Type":"ContainerStarted","Data":"b7e3a55a312f286fc2e120e68ee82dc0db0356cc632fb2b5d896fdbf17d1d384"}
Jul 23 00:28:10.803608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:10.803568    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hc78w" podStartSLOduration=33.597598632 podStartE2EDuration="45.8035523s" totalImagesPullingTime="12.205953668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:27:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:28:10.802038222 +0000 UTC m=+294.608449452" watchObservedRunningTime="2026-07-23 00:28:10.8035523 +0000 UTC m=+294.609963559"
Jul 23 00:28:11.788234 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:11.788199    2561 generic.go:356] "Generic (PLEG): container finished" podID="a7e819d6-e8d1-47ec-acfc-d58fb660dc66" containerID="ee723481c397e59de6b06617453ae67e043f242decdd492e1b4f13bf91161864" exitCode=0
Jul 23 00:28:11.788702 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:11.788285    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wf4m9" event={"ID":"a7e819d6-e8d1-47ec-acfc-d58fb660dc66","Type":"ContainerDied","Data":"ee723481c397e59de6b06617453ae67e043f242decdd492e1b4f13bf91161864"}
Jul 23 00:28:12.792943 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:12.792905    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wf4m9" event={"ID":"a7e819d6-e8d1-47ec-acfc-d58fb660dc66","Type":"ContainerStarted","Data":"a2f6e93ef1c0adc95e6eef44a23ad305f270059677ee356f8e7bf744adf289df"}
Jul 23 00:28:12.793330 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:12.792967    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wf4m9" event={"ID":"a7e819d6-e8d1-47ec-acfc-d58fb660dc66","Type":"ContainerStarted","Data":"20220d3fc6831d809b14aa14af9010d5b883ea3639658927fa8c6936dac9964b"}
Jul 23 00:28:12.815444 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:12.815396    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wf4m9" podStartSLOduration=3.914279534 podStartE2EDuration="4.815381044s" totalImagesPullingTime="837.030761ms" totalInitContainerRuntime="64.070749ms" isStatefulPod=true podCreationTimestamp="2026-07-23 00:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:28:12.813686835 +0000 UTC m=+296.620098068" watchObservedRunningTime="2026-07-23 00:28:12.815381044 +0000 UTC m=+296.621792268"
Jul 23 00:28:14.783000 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:28:14.782957    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6dn2c" podUID="7c65b730-bf89-4090-b8a7-cad9a31bf810"
Jul 23 00:28:16.652003 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:16.651974    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:28:16.653086 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:16.653067    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:28:16.655296 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:16.655278    2561 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 00:28:16.768514 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:16.768486    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jgpkp"
Jul 23 00:28:18.602022 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:18.601987    2561 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs" podUID="d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 23 00:28:20.782563 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:20.782532    2561 scope.go:117] "RemoveContainer" containerID="3c663ee834b0c99e0cc7f58094df64e3a0ebbcd217d0ac1d2ff1a4cb0fdc67f1"
Jul 23 00:28:21.824645 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:21.824608    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65" event={"ID":"cd2d8df2-d752-4524-991f-aa1def67e57e","Type":"ContainerStarted","Data":"5aa404427e3482f696e22b500817a8474db376e73b3e77199c6741dfdb1d53fb"}
Jul 23 00:28:21.825150 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:21.824889    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:28:21.826406 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:21.826384    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c5ddd655b-ksf65"
Jul 23 00:28:22.329958 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:22.329928    2561 patch_prober.go:28] interesting pod/image-registry-868df8cc95-gjff5 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 23 00:28:22.330116 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:22.329975    2561 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-868df8cc95-gjff5" podUID="6b5feda6-e797-4219-9e8a-bc0f0a26909f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 00:28:22.781473 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:22.781444    2561 scope.go:117] "RemoveContainer" containerID="cdd56cc3f5eecc3d803db440755599c8a89d144c15091f825ed153e7b68df69c"
Jul 23 00:28:23.556554 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:23.556522    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-868df8cc95-gjff5"]
Jul 23 00:28:23.560793 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:23.560764    2561 patch_prober.go:28] interesting pod/image-registry-868df8cc95-gjff5 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 23 00:28:23.560921 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:23.560813    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-868df8cc95-gjff5" podUID="6b5feda6-e797-4219-9e8a-bc0f0a26909f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 00:28:23.835419 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:23.835330    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cb57d4b66-j974k" event={"ID":"2d9c04b3-b899-4884-9e89-0037f29f8a0f","Type":"ContainerStarted","Data":"d7c5816f56a3bf49749c7ea250f33ec10c60c16a48bcbacd258b73368af1ddb0"}
Jul 23 00:28:28.601808 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:28.601767    2561 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs" podUID="d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 23 00:28:28.782139 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:28.782107    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:28:33.560690 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:33.560662    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:28:38.601800 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:38.601765    2561 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs" podUID="d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 23 00:28:38.602202 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:38.601838    2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs"
Jul 23 00:28:38.602291 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:38.602261    2561 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"449280d3c2f6d6887a14acaf8607b75ac7d9678becc8da24f0143382da6c61a4"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 23 00:28:38.602327 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:38.602310    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs" podUID="d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e" containerName="service-proxy" containerID="cri-o://449280d3c2f6d6887a14acaf8607b75ac7d9678becc8da24f0143382da6c61a4" gracePeriod=30
Jul 23 00:28:38.884191 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:38.884106    2561 generic.go:356] "Generic (PLEG): container finished" podID="d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e" containerID="449280d3c2f6d6887a14acaf8607b75ac7d9678becc8da24f0143382da6c61a4" exitCode=2
Jul 23 00:28:38.884191 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:38.884164    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs" event={"ID":"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e","Type":"ContainerDied","Data":"449280d3c2f6d6887a14acaf8607b75ac7d9678becc8da24f0143382da6c61a4"}
Jul 23 00:28:38.884396 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:38.884197    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b8c8d4c6f-9lwrs" event={"ID":"d5eb4f62-b8d0-48bf-aa4a-a2c48646dd0e","Type":"ContainerStarted","Data":"50044783ad0e85632176e241a6ba619b5e0be51005f2620f0784b4e26924cc2f"}
Jul 23 00:28:48.576095 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.576034    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-868df8cc95-gjff5" podUID="6b5feda6-e797-4219-9e8a-bc0f0a26909f" containerName="registry" containerID="cri-o://abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720" gracePeriod=30
Jul 23 00:28:48.816780 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.816759    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:28:48.879529 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.879456    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") pod \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") "
Jul 23 00:28:48.879529 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.879491    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b5feda6-e797-4219-9e8a-bc0f0a26909f-ca-trust-extracted\") pod \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") "
Jul 23 00:28:48.879529 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.879515    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-bound-sa-token\") pod \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") "
Jul 23 00:28:48.879836 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.879547    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b5feda6-e797-4219-9e8a-bc0f0a26909f-installation-pull-secrets\") pod \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") "
Jul 23 00:28:48.879836 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.879657    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b5feda6-e797-4219-9e8a-bc0f0a26909f-image-registry-private-configuration\") pod \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") "
Jul 23 00:28:48.879836 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.879710    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5dbp\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-kube-api-access-f5dbp\") pod \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") "
Jul 23 00:28:48.879836 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.879752    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-certificates\") pod \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") "
Jul 23 00:28:48.879836 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.879779    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b5feda6-e797-4219-9e8a-bc0f0a26909f-trusted-ca\") pod \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\" (UID: \"6b5feda6-e797-4219-9e8a-bc0f0a26909f\") "
Jul 23 00:28:48.880429 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.880399    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6b5feda6-e797-4219-9e8a-bc0f0a26909f" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:28:48.880551 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.880408    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b5feda6-e797-4219-9e8a-bc0f0a26909f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6b5feda6-e797-4219-9e8a-bc0f0a26909f" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:28:48.882463 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.882407    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6b5feda6-e797-4219-9e8a-bc0f0a26909f" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:28:48.882463 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.882437    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6b5feda6-e797-4219-9e8a-bc0f0a26909f" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:28:48.882463 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.882434    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b5feda6-e797-4219-9e8a-bc0f0a26909f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6b5feda6-e797-4219-9e8a-bc0f0a26909f" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:28:48.882702 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.882634    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b5feda6-e797-4219-9e8a-bc0f0a26909f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6b5feda6-e797-4219-9e8a-bc0f0a26909f" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:28:48.882761 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.882699    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-kube-api-access-f5dbp" (OuterVolumeSpecName: "kube-api-access-f5dbp") pod "6b5feda6-e797-4219-9e8a-bc0f0a26909f" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f"). InnerVolumeSpecName "kube-api-access-f5dbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:28:48.889007 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.888982    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b5feda6-e797-4219-9e8a-bc0f0a26909f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6b5feda6-e797-4219-9e8a-bc0f0a26909f" (UID: "6b5feda6-e797-4219-9e8a-bc0f0a26909f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:28:48.917599 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.917569    2561 generic.go:356] "Generic (PLEG): container finished" podID="6b5feda6-e797-4219-9e8a-bc0f0a26909f" containerID="abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720" exitCode=0
Jul 23 00:28:48.917724 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.917630    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868df8cc95-gjff5"
Jul 23 00:28:48.917724 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.917646    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868df8cc95-gjff5" event={"ID":"6b5feda6-e797-4219-9e8a-bc0f0a26909f","Type":"ContainerDied","Data":"abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720"}
Jul 23 00:28:48.917724 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.917708    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868df8cc95-gjff5" event={"ID":"6b5feda6-e797-4219-9e8a-bc0f0a26909f","Type":"ContainerDied","Data":"59ce9f1a1935a3d0923281fee1bafba1f38dc7c8888221656ab118bc891b2afe"}
Jul 23 00:28:48.917831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.917728    2561 scope.go:117] "RemoveContainer" containerID="abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720"
Jul 23 00:28:48.926685 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.926668    2561 scope.go:117] "RemoveContainer" containerID="abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720"
Jul 23 00:28:48.926950 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:28:48.926931    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720\": container with ID starting with abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720 not found: ID does not exist" containerID="abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720"
Jul 23 00:28:48.927008 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.926958    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720"} err="failed to get container status \"abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720\": rpc error: code = NotFound desc = could not find container \"abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720\": container with ID starting with abf04221dab4804019662134f71df4c30060aff758409a490cc31576f9d6b720 not found: ID does not exist"
Jul 23 00:28:48.940235 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.940212    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-868df8cc95-gjff5"]
Jul 23 00:28:48.943710 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.943690    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-868df8cc95-gjff5"]
Jul 23 00:28:48.981009 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.980982    2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-bound-sa-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:28:48.981009 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.981006    2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6b5feda6-e797-4219-9e8a-bc0f0a26909f-installation-pull-secrets\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:28:48.981128 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.981016    2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6b5feda6-e797-4219-9e8a-bc0f0a26909f-image-registry-private-configuration\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:28:48.981128 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.981026    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5dbp\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-kube-api-access-f5dbp\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:28:48.981128 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.981035    2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-certificates\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:28:48.981128 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.981043    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6b5feda6-e797-4219-9e8a-bc0f0a26909f-trusted-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:28:48.981128 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.981051    2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6b5feda6-e797-4219-9e8a-bc0f0a26909f-registry-tls\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:28:48.981128 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:48.981059    2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6b5feda6-e797-4219-9e8a-bc0f0a26909f-ca-trust-extracted\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:28:50.785784 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:28:50.785744    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b5feda6-e797-4219-9e8a-bc0f0a26909f" path="/var/lib/kubelet/pods/6b5feda6-e797-4219-9e8a-bc0f0a26909f/volumes"
Jul 23 00:29:30.575399 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:29:30.575342    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:29:30.577666 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:29:30.577634    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c65b730-bf89-4090-b8a7-cad9a31bf810-metrics-certs\") pod \"network-metrics-daemon-6dn2c\" (UID: \"7c65b730-bf89-4090-b8a7-cad9a31bf810\") " pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:29:30.585059 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:29:30.585040    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8gvmb\""
Jul 23 00:29:30.593141 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:29:30.593115    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dn2c"
Jul 23 00:29:30.716806 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:29:30.716775    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6dn2c"]
Jul 23 00:29:30.719454 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:29:30.719424    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c65b730_bf89_4090_b8a7_cad9a31bf810.slice/crio-a892750173d69cf963ef1d45c22e77631cedeca258d5ece1f0566f38de0cc117 WatchSource:0}: Error finding container a892750173d69cf963ef1d45c22e77631cedeca258d5ece1f0566f38de0cc117: Status 404 returned error can't find the container with id a892750173d69cf963ef1d45c22e77631cedeca258d5ece1f0566f38de0cc117
Jul 23 00:29:30.721455 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:29:30.721435    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 00:29:31.058377 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:29:31.058329    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6dn2c" event={"ID":"7c65b730-bf89-4090-b8a7-cad9a31bf810","Type":"ContainerStarted","Data":"a892750173d69cf963ef1d45c22e77631cedeca258d5ece1f0566f38de0cc117"}
Jul 23 00:29:32.063141 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:29:32.063054    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6dn2c" event={"ID":"7c65b730-bf89-4090-b8a7-cad9a31bf810","Type":"ContainerStarted","Data":"bba124ab9d81066c440eef5d262f4090961a7e2f50799c568faaea795b60101f"}
Jul 23 00:29:32.063141 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:29:32.063112    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6dn2c" event={"ID":"7c65b730-bf89-4090-b8a7-cad9a31bf810","Type":"ContainerStarted","Data":"8ae1d3987bbc2fbfb2572a1fb331b31235ca6c9e0cf02295a210bf529be12cf8"}
Jul 23 00:29:32.079706 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:29:32.079659    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6dn2c" podStartSLOduration=374.057531762 podStartE2EDuration="6m15.079644817s" totalImagesPullingTime="1.022113055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:29:32.078861683 +0000 UTC m=+375.885272917" watchObservedRunningTime="2026-07-23 00:29:32.079644817 +0000 UTC m=+375.886056049"
Jul 23 00:32:34.102654 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.102572    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999"]
Jul 23 00:32:34.103118 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.102787    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b5feda6-e797-4219-9e8a-bc0f0a26909f" containerName="registry"
Jul 23 00:32:34.103118 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.102797    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b5feda6-e797-4219-9e8a-bc0f0a26909f" containerName="registry"
Jul 23 00:32:34.103118 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.102846    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b5feda6-e797-4219-9e8a-bc0f0a26909f" containerName="registry"
Jul 23 00:32:34.105553 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.105538    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999"
Jul 23 00:32:34.108262 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.108229    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-c4pkc\""
Jul 23 00:32:34.108412 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.108278    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 23 00:32:34.109181 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.109157    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 23 00:32:34.113842 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.113822    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999"]
Jul 23 00:32:34.296506 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.296476    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10162be2-0f9c-45f9-a3a9-8dfc01e2e241-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cf999\" (UID: \"10162be2-0f9c-45f9-a3a9-8dfc01e2e241\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999"
Jul 23 00:32:34.296690 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.296528    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j8wc\" (UniqueName: \"kubernetes.io/projected/10162be2-0f9c-45f9-a3a9-8dfc01e2e241-kube-api-access-7j8wc\") pod \"openshift-lws-operator-fd8ccff4c-cf999\" (UID: \"10162be2-0f9c-45f9-a3a9-8dfc01e2e241\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999"
Jul 23 00:32:34.397665 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.397597    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10162be2-0f9c-45f9-a3a9-8dfc01e2e241-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cf999\" (UID: \"10162be2-0f9c-45f9-a3a9-8dfc01e2e241\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999"
Jul 23 00:32:34.397665 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.397644    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j8wc\" (UniqueName: \"kubernetes.io/projected/10162be2-0f9c-45f9-a3a9-8dfc01e2e241-kube-api-access-7j8wc\") pod \"openshift-lws-operator-fd8ccff4c-cf999\" (UID: \"10162be2-0f9c-45f9-a3a9-8dfc01e2e241\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999"
Jul 23 00:32:34.398049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.398026    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10162be2-0f9c-45f9-a3a9-8dfc01e2e241-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cf999\" (UID: \"10162be2-0f9c-45f9-a3a9-8dfc01e2e241\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999"
Jul 23 00:32:34.406102 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.406078    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j8wc\" (UniqueName: \"kubernetes.io/projected/10162be2-0f9c-45f9-a3a9-8dfc01e2e241-kube-api-access-7j8wc\") pod \"openshift-lws-operator-fd8ccff4c-cf999\" (UID: \"10162be2-0f9c-45f9-a3a9-8dfc01e2e241\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999"
Jul 23 00:32:34.413938 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.413916    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999"
Jul 23 00:32:34.538415 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.536528    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999"]
Jul 23 00:32:34.541827 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:32:34.541803    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10162be2_0f9c_45f9_a3a9_8dfc01e2e241.slice/crio-d0733bfe0b80208dbea143953ae122de4e6b465e6debbc6db174b44286a633ba WatchSource:0}: Error finding container d0733bfe0b80208dbea143953ae122de4e6b465e6debbc6db174b44286a633ba: Status 404 returned error can't find the container with id d0733bfe0b80208dbea143953ae122de4e6b465e6debbc6db174b44286a633ba
Jul 23 00:32:34.655611 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:34.655514    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999" event={"ID":"10162be2-0f9c-45f9-a3a9-8dfc01e2e241","Type":"ContainerStarted","Data":"d0733bfe0b80208dbea143953ae122de4e6b465e6debbc6db174b44286a633ba"}
Jul 23 00:32:37.669382 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:37.669337    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999" event={"ID":"10162be2-0f9c-45f9-a3a9-8dfc01e2e241","Type":"ContainerStarted","Data":"f83dfe998fec9d44c335c61ba142b884d1b1d977704bf9ee6a5ac894dfa84b72"}
Jul 23 00:32:37.687312 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:37.687262    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cf999" podStartSLOduration=1.280821456 podStartE2EDuration="3.687245155s" totalImagesPullingTime="2.406423699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:32:37.686548975 +0000 UTC m=+561.492960209" watchObservedRunningTime="2026-07-23 00:32:37.687245155 +0000 UTC m=+561.493656387"
Jul 23 00:32:55.666161 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.666125    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"]
Jul 23 00:32:55.669158 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.669142    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.672427 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.672405    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 00:32:55.672572 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.672440    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 00:32:55.672572 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.672490    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 00:32:55.672669 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.672592    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 00:32:55.672715 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.672692    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hhnpz\""
Jul 23 00:32:55.689116 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.689093    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"]
Jul 23 00:32:55.743021 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.742989    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16466a8f-10c0-4a8e-a961-221d79f2b668-webhook-cert\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.743184 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.743027    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt8gc\" (UniqueName: \"kubernetes.io/projected/16466a8f-10c0-4a8e-a961-221d79f2b668-kube-api-access-wt8gc\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.743184 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.743053    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/16466a8f-10c0-4a8e-a961-221d79f2b668-manifests\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.743184 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.743105    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16466a8f-10c0-4a8e-a961-221d79f2b668-tmp\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.743184 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.743140    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/16466a8f-10c0-4a8e-a961-221d79f2b668-apiservice-cert\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.844017 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.843979    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16466a8f-10c0-4a8e-a961-221d79f2b668-webhook-cert\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.844017 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.844017    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt8gc\" (UniqueName: \"kubernetes.io/projected/16466a8f-10c0-4a8e-a961-221d79f2b668-kube-api-access-wt8gc\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.844287 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.844036    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/16466a8f-10c0-4a8e-a961-221d79f2b668-manifests\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.844287 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.844064    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16466a8f-10c0-4a8e-a961-221d79f2b668-tmp\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.844287 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.844098    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/16466a8f-10c0-4a8e-a961-221d79f2b668-apiservice-cert\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.844582 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.844558    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/16466a8f-10c0-4a8e-a961-221d79f2b668-manifests\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.846555 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.846536    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16466a8f-10c0-4a8e-a961-221d79f2b668-tmp\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.846761 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.846741    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/16466a8f-10c0-4a8e-a961-221d79f2b668-apiservice-cert\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.846804 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.846741    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16466a8f-10c0-4a8e-a961-221d79f2b668-webhook-cert\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.856178 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.856156    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt8gc\" (UniqueName: \"kubernetes.io/projected/16466a8f-10c0-4a8e-a961-221d79f2b668-kube-api-access-wt8gc\") pod \"opendatahub-operator-controller-manager-695645577-bn7qh\" (UID: \"16466a8f-10c0-4a8e-a961-221d79f2b668\") " pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:55.979520 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:55.979476    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:32:56.108183 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:56.108161    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"]
Jul 23 00:32:56.110742 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:32:56.110711    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16466a8f_10c0_4a8e_a961_221d79f2b668.slice/crio-f4dda2eea4753c12e2d7dc44bc266db8f1a5ef75e5857df041c01349f3d39bb6 WatchSource:0}: Error finding container f4dda2eea4753c12e2d7dc44bc266db8f1a5ef75e5857df041c01349f3d39bb6: Status 404 returned error can't find the container with id f4dda2eea4753c12e2d7dc44bc266db8f1a5ef75e5857df041c01349f3d39bb6
Jul 23 00:32:56.735545 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:56.735502    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh" event={"ID":"16466a8f-10c0-4a8e-a961-221d79f2b668","Type":"ContainerStarted","Data":"f4dda2eea4753c12e2d7dc44bc266db8f1a5ef75e5857df041c01349f3d39bb6"}
Jul 23 00:32:59.747788 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:32:59.747757    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh" event={"ID":"16466a8f-10c0-4a8e-a961-221d79f2b668","Type":"ContainerStarted","Data":"efa5bd09298f8cca6ecacfa0541c5ca36d5f4d27a6e9be588f53e57e6a884e46"}
Jul 23 00:33:00.752285 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:00.752252    2561 generic.go:356] "Generic (PLEG): container finished" podID="16466a8f-10c0-4a8e-a961-221d79f2b668" containerID="efa5bd09298f8cca6ecacfa0541c5ca36d5f4d27a6e9be588f53e57e6a884e46" exitCode=0
Jul 23 00:33:00.752769 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:00.752296    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh" event={"ID":"16466a8f-10c0-4a8e-a961-221d79f2b668","Type":"ContainerDied","Data":"efa5bd09298f8cca6ecacfa0541c5ca36d5f4d27a6e9be588f53e57e6a884e46"}
Jul 23 00:33:01.758207 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:01.758180    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh" event={"ID":"16466a8f-10c0-4a8e-a961-221d79f2b668","Type":"ContainerStarted","Data":"6c3c5f651e44e9cf56e8ebcc2e4b93c2c7e00ca5d67e0c17bc7d061de4f792e3"}
Jul 23 00:33:01.758609 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:01.758307    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:33:01.795847 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:01.795788    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh" podStartSLOduration=1.610347966 podStartE2EDuration="6.79576776s" totalImagesPullingTime="4.128039187s" totalInitContainerRuntime="1.057380607s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:32:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:33:01.793547834 +0000 UTC m=+585.599959068" watchObservedRunningTime="2026-07-23 00:33:01.79576776 +0000 UTC m=+585.602178993"
Jul 23 00:33:12.765486 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:12.765456    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-695645577-bn7qh"
Jul 23 00:33:15.823135 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.823106    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"]
Jul 23 00:33:15.835351 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.835324    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"
Jul 23 00:33:15.838319 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.838296    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xpwtf\""
Jul 23 00:33:15.839492 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.839471    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 23 00:33:15.839492 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.839471    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 00:33:15.839693 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.839547    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 00:33:15.839693 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.839648    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 00:33:15.850021 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.849999    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"]
Jul 23 00:33:15.885350 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.885324    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clb6r\" (UniqueName: \"kubernetes.io/projected/360bf7a3-397c-47be-b280-da4c534f26dd-kube-api-access-clb6r\") pod \"kube-auth-proxy-678df7485d-fgqcj\" (UID: \"360bf7a3-397c-47be-b280-da4c534f26dd\") " pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"
Jul 23 00:33:15.885489 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.885353    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/360bf7a3-397c-47be-b280-da4c534f26dd-tls-certs\") pod \"kube-auth-proxy-678df7485d-fgqcj\" (UID: \"360bf7a3-397c-47be-b280-da4c534f26dd\") " pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"
Jul 23 00:33:15.885489 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.885399    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/360bf7a3-397c-47be-b280-da4c534f26dd-tmp\") pod \"kube-auth-proxy-678df7485d-fgqcj\" (UID: \"360bf7a3-397c-47be-b280-da4c534f26dd\") " pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"
Jul 23 00:33:15.986422 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.986352    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clb6r\" (UniqueName: \"kubernetes.io/projected/360bf7a3-397c-47be-b280-da4c534f26dd-kube-api-access-clb6r\") pod \"kube-auth-proxy-678df7485d-fgqcj\" (UID: \"360bf7a3-397c-47be-b280-da4c534f26dd\") " pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"
Jul 23 00:33:15.986422 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.986436    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/360bf7a3-397c-47be-b280-da4c534f26dd-tls-certs\") pod \"kube-auth-proxy-678df7485d-fgqcj\" (UID: \"360bf7a3-397c-47be-b280-da4c534f26dd\") " pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"
Jul 23 00:33:15.986624 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.986465    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/360bf7a3-397c-47be-b280-da4c534f26dd-tmp\") pod \"kube-auth-proxy-678df7485d-fgqcj\" (UID: \"360bf7a3-397c-47be-b280-da4c534f26dd\") " pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"
Jul 23 00:33:15.988792 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.988771    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/360bf7a3-397c-47be-b280-da4c534f26dd-tmp\") pod \"kube-auth-proxy-678df7485d-fgqcj\" (UID: \"360bf7a3-397c-47be-b280-da4c534f26dd\") " pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"
Jul 23 00:33:15.989003 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.988986    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/360bf7a3-397c-47be-b280-da4c534f26dd-tls-certs\") pod \"kube-auth-proxy-678df7485d-fgqcj\" (UID: \"360bf7a3-397c-47be-b280-da4c534f26dd\") " pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"
Jul 23 00:33:15.997216 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:15.997197    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clb6r\" (UniqueName: \"kubernetes.io/projected/360bf7a3-397c-47be-b280-da4c534f26dd-kube-api-access-clb6r\") pod \"kube-auth-proxy-678df7485d-fgqcj\" (UID: \"360bf7a3-397c-47be-b280-da4c534f26dd\") " pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"
Jul 23 00:33:16.144358 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:16.144288    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"
Jul 23 00:33:16.280277 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:16.280254    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-678df7485d-fgqcj"]
Jul 23 00:33:16.281978 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:33:16.281947    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod360bf7a3_397c_47be_b280_da4c534f26dd.slice/crio-a2e2161a5198332edfb99f5f488dc5e388bd693a3b4dac5370cbcf64dff977b6 WatchSource:0}: Error finding container a2e2161a5198332edfb99f5f488dc5e388bd693a3b4dac5370cbcf64dff977b6: Status 404 returned error can't find the container with id a2e2161a5198332edfb99f5f488dc5e388bd693a3b4dac5370cbcf64dff977b6
Jul 23 00:33:16.674132 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:16.674109    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:33:16.674300 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:16.674194    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:33:16.814559 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:16.814524    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj" event={"ID":"360bf7a3-397c-47be-b280-da4c534f26dd","Type":"ContainerStarted","Data":"a2e2161a5198332edfb99f5f488dc5e388bd693a3b4dac5370cbcf64dff977b6"}
Jul 23 00:33:17.145516 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:17.145480    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qqlgk"]
Jul 23 00:33:17.147708 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:17.147687    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk"
Jul 23 00:33:17.150257 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:17.150236    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 23 00:33:17.150389 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:17.150237    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-4ckkw\""
Jul 23 00:33:17.166209 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:17.166155    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qqlgk"]
Jul 23 00:33:17.196966 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:17.196933    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5dhf\" (UniqueName: \"kubernetes.io/projected/bc9b2386-9099-45b6-bc22-bbe13ae267c8-kube-api-access-g5dhf\") pod \"model-serving-api-54d89bd5fc-qqlgk\" (UID: \"bc9b2386-9099-45b6-bc22-bbe13ae267c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk"
Jul 23 00:33:17.197122 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:17.196987    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc9b2386-9099-45b6-bc22-bbe13ae267c8-tls-certs\") pod \"model-serving-api-54d89bd5fc-qqlgk\" (UID: \"bc9b2386-9099-45b6-bc22-bbe13ae267c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk"
Jul 23 00:33:17.298370 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:17.298336    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc9b2386-9099-45b6-bc22-bbe13ae267c8-tls-certs\") pod \"model-serving-api-54d89bd5fc-qqlgk\" (UID: \"bc9b2386-9099-45b6-bc22-bbe13ae267c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk"
Jul 23 00:33:17.298548 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:17.298443    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5dhf\" (UniqueName: \"kubernetes.io/projected/bc9b2386-9099-45b6-bc22-bbe13ae267c8-kube-api-access-g5dhf\") pod \"model-serving-api-54d89bd5fc-qqlgk\" (UID: \"bc9b2386-9099-45b6-bc22-bbe13ae267c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk"
Jul 23 00:33:17.298548 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:33:17.298512    2561 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 23 00:33:17.298656 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:33:17.298580    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc9b2386-9099-45b6-bc22-bbe13ae267c8-tls-certs podName:bc9b2386-9099-45b6-bc22-bbe13ae267c8 nodeName:}" failed. No retries permitted until 2026-07-23 00:33:17.79855827 +0000 UTC m=+601.604969486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bc9b2386-9099-45b6-bc22-bbe13ae267c8-tls-certs") pod "model-serving-api-54d89bd5fc-qqlgk" (UID: "bc9b2386-9099-45b6-bc22-bbe13ae267c8") : secret "model-serving-api-tls" not found
Jul 23 00:33:17.339170 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:17.339138    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5dhf\" (UniqueName: \"kubernetes.io/projected/bc9b2386-9099-45b6-bc22-bbe13ae267c8-kube-api-access-g5dhf\") pod \"model-serving-api-54d89bd5fc-qqlgk\" (UID: \"bc9b2386-9099-45b6-bc22-bbe13ae267c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk"
Jul 23 00:33:17.802447 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:17.802408    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc9b2386-9099-45b6-bc22-bbe13ae267c8-tls-certs\") pod \"model-serving-api-54d89bd5fc-qqlgk\" (UID: \"bc9b2386-9099-45b6-bc22-bbe13ae267c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk"
Jul 23 00:33:17.802658 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:33:17.802570    2561 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 23 00:33:17.802658 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:33:17.802647    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc9b2386-9099-45b6-bc22-bbe13ae267c8-tls-certs podName:bc9b2386-9099-45b6-bc22-bbe13ae267c8 nodeName:}" failed. No retries permitted until 2026-07-23 00:33:18.802627549 +0000 UTC m=+602.609038763 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bc9b2386-9099-45b6-bc22-bbe13ae267c8-tls-certs") pod "model-serving-api-54d89bd5fc-qqlgk" (UID: "bc9b2386-9099-45b6-bc22-bbe13ae267c8") : secret "model-serving-api-tls" not found
Jul 23 00:33:18.810052 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:18.810017    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc9b2386-9099-45b6-bc22-bbe13ae267c8-tls-certs\") pod \"model-serving-api-54d89bd5fc-qqlgk\" (UID: \"bc9b2386-9099-45b6-bc22-bbe13ae267c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk"
Jul 23 00:33:18.812614 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:18.812591    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc9b2386-9099-45b6-bc22-bbe13ae267c8-tls-certs\") pod \"model-serving-api-54d89bd5fc-qqlgk\" (UID: \"bc9b2386-9099-45b6-bc22-bbe13ae267c8\") " pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk"
Jul 23 00:33:18.956246 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:18.956224    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 00:33:18.961469 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:18.961450    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk"
Jul 23 00:33:19.088958 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:19.088936    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qqlgk"]
Jul 23 00:33:19.091381 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:33:19.091346    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc9b2386_9099_45b6_bc22_bbe13ae267c8.slice/crio-41d6b990b46f61c5083330539044f7253ebf14c8c994b7c99a28b708d1d3017a WatchSource:0}: Error finding container 41d6b990b46f61c5083330539044f7253ebf14c8c994b7c99a28b708d1d3017a: Status 404 returned error can't find the container with id 41d6b990b46f61c5083330539044f7253ebf14c8c994b7c99a28b708d1d3017a
Jul 23 00:33:19.827252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:19.827219    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk" event={"ID":"bc9b2386-9099-45b6-bc22-bbe13ae267c8","Type":"ContainerStarted","Data":"41d6b990b46f61c5083330539044f7253ebf14c8c994b7c99a28b708d1d3017a"}
Jul 23 00:33:19.828658 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:19.828635    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj" event={"ID":"360bf7a3-397c-47be-b280-da4c534f26dd","Type":"ContainerStarted","Data":"85f10cf4733f3f3a75683db9bdbc4c4d2433523c644c4609868fe6d1eb799d7a"}
Jul 23 00:33:19.848405 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:19.848340    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-678df7485d-fgqcj" podStartSLOduration=2.178659798 podStartE2EDuration="4.848325467s" totalImagesPullingTime="2.669665669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:33:19.847025167 +0000 UTC m=+603.653436410" watchObservedRunningTime="2026-07-23 00:33:19.848325467 +0000 UTC m=+603.654736759"
Jul 23 00:33:21.839112 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:21.839081    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk" event={"ID":"bc9b2386-9099-45b6-bc22-bbe13ae267c8","Type":"ContainerStarted","Data":"ed0b7fa7688bd1b9ea52828165a9971b13863b7b2bd26c7eb12a7f55493fa5fe"}
Jul 23 00:33:21.839570 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:21.839198    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk"
Jul 23 00:33:21.860564 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:21.860514    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk" podStartSLOduration=3.17498823 podStartE2EDuration="4.860491933s" totalImagesPullingTime="1.685503703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:33:21.860247734 +0000 UTC m=+605.666658968" watchObservedRunningTime="2026-07-23 00:33:21.860491933 +0000 UTC m=+605.666903169"
Jul 23 00:33:24.281875 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.281845    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"]
Jul 23 00:33:24.283770 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.283753    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.288192 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.288171    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 23 00:33:24.288192 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.288179    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 23 00:33:24.288390 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.288199    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 23 00:33:24.288390 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.288206    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 23 00:33:24.288484 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.288472    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-tthvl\""
Jul 23 00:33:24.305533 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.303569    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"]
Jul 23 00:33:24.352523 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.352496    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.352677 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.352529    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.352677 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.352562    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.352677 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.352649    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.352784 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.352683    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.352784 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.352714    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.352784 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.352771    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdcfc\" (UniqueName: \"kubernetes.io/projected/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-kube-api-access-vdcfc\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.453714 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.453681    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.453714 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.453713    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.453895 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.453739    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.453895 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.453758    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdcfc\" (UniqueName: \"kubernetes.io/projected/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-kube-api-access-vdcfc\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.453970 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.453904    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.453970 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.453944    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.454071 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.453992    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.454835 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.454804    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.456105 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.456071    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.456191 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.456083    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.456191 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.456181    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.456265 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.456206    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.462226 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.462196    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.462319 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.462298    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdcfc\" (UniqueName: \"kubernetes.io/projected/760e166b-3e6c-47c6-b2f6-3ab7b4180f53-kube-api-access-vdcfc\") pod \"istiod-openshift-gateway-94bb8fbfd-g2w67\" (UID: \"760e166b-3e6c-47c6-b2f6-3ab7b4180f53\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.592914 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.592831    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:24.729106 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.729082    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"]
Jul 23 00:33:24.731079 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:33:24.731050    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod760e166b_3e6c_47c6_b2f6_3ab7b4180f53.slice/crio-840b180bffd316691417a7405811f5b605951e702e514005820e429acd364627 WatchSource:0}: Error finding container 840b180bffd316691417a7405811f5b605951e702e514005820e429acd364627: Status 404 returned error can't find the container with id 840b180bffd316691417a7405811f5b605951e702e514005820e429acd364627
Jul 23 00:33:24.850941 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:24.850857    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67" event={"ID":"760e166b-3e6c-47c6-b2f6-3ab7b4180f53","Type":"ContainerStarted","Data":"840b180bffd316691417a7405811f5b605951e702e514005820e429acd364627"}
Jul 23 00:33:27.019902 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.019863    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"]
Jul 23 00:33:27.022005 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.021982    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"
Jul 23 00:33:27.024726 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.024702    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 23 00:33:27.025592 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.025550    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-z8skc\""
Jul 23 00:33:27.026122 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.026100    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"]
Jul 23 00:33:27.028138 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.028117    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"
Jul 23 00:33:27.030509 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.030485    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 23 00:33:27.030917 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.030647    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-tsl6x\""
Jul 23 00:33:27.035559 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.035540    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"]
Jul 23 00:33:27.042608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.042579    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"]
Jul 23 00:33:27.075017 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.074978    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhlj4\" (UniqueName: \"kubernetes.io/projected/816a95a0-4fa8-48ff-9486-45ac658ee1f2-kube-api-access-lhlj4\") pod \"kserve-controller-manager-d6f4b65b8-kh6dp\" (UID: \"816a95a0-4fa8-48ff-9486-45ac658ee1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"
Jul 23 00:33:27.075188 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.075031    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/816a95a0-4fa8-48ff-9486-45ac658ee1f2-cert\") pod \"kserve-controller-manager-d6f4b65b8-kh6dp\" (UID: \"816a95a0-4fa8-48ff-9486-45ac658ee1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"
Jul 23 00:33:27.176506 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.176467    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chg5l\" (UniqueName: \"kubernetes.io/projected/c5f8583e-370b-4998-a270-4e09f3bb2722-kube-api-access-chg5l\") pod \"llmisvc-controller-manager-777c6bd7d5-rnpwl\" (UID: \"c5f8583e-370b-4998-a270-4e09f3bb2722\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"
Jul 23 00:33:27.176685 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.176541    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhlj4\" (UniqueName: \"kubernetes.io/projected/816a95a0-4fa8-48ff-9486-45ac658ee1f2-kube-api-access-lhlj4\") pod \"kserve-controller-manager-d6f4b65b8-kh6dp\" (UID: \"816a95a0-4fa8-48ff-9486-45ac658ee1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"
Jul 23 00:33:27.176685 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.176579    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/816a95a0-4fa8-48ff-9486-45ac658ee1f2-cert\") pod \"kserve-controller-manager-d6f4b65b8-kh6dp\" (UID: \"816a95a0-4fa8-48ff-9486-45ac658ee1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"
Jul 23 00:33:27.176779 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:33:27.176688    2561 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 23 00:33:27.176779 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.176707    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5f8583e-370b-4998-a270-4e09f3bb2722-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rnpwl\" (UID: \"c5f8583e-370b-4998-a270-4e09f3bb2722\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"
Jul 23 00:33:27.176779 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:33:27.176751    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/816a95a0-4fa8-48ff-9486-45ac658ee1f2-cert podName:816a95a0-4fa8-48ff-9486-45ac658ee1f2 nodeName:}" failed. No retries permitted until 2026-07-23 00:33:27.676730315 +0000 UTC m=+611.483141530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/816a95a0-4fa8-48ff-9486-45ac658ee1f2-cert") pod "kserve-controller-manager-d6f4b65b8-kh6dp" (UID: "816a95a0-4fa8-48ff-9486-45ac658ee1f2") : secret "kserve-webhook-server-cert" not found
Jul 23 00:33:27.193467 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.193438    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhlj4\" (UniqueName: \"kubernetes.io/projected/816a95a0-4fa8-48ff-9486-45ac658ee1f2-kube-api-access-lhlj4\") pod \"kserve-controller-manager-d6f4b65b8-kh6dp\" (UID: \"816a95a0-4fa8-48ff-9486-45ac658ee1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"
Jul 23 00:33:27.277622 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.277541    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chg5l\" (UniqueName: \"kubernetes.io/projected/c5f8583e-370b-4998-a270-4e09f3bb2722-kube-api-access-chg5l\") pod \"llmisvc-controller-manager-777c6bd7d5-rnpwl\" (UID: \"c5f8583e-370b-4998-a270-4e09f3bb2722\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"
Jul 23 00:33:27.277763 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.277718    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5f8583e-370b-4998-a270-4e09f3bb2722-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rnpwl\" (UID: \"c5f8583e-370b-4998-a270-4e09f3bb2722\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"
Jul 23 00:33:27.277850 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:33:27.277835    2561 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 23 00:33:27.277905 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:33:27.277900    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5f8583e-370b-4998-a270-4e09f3bb2722-cert podName:c5f8583e-370b-4998-a270-4e09f3bb2722 nodeName:}" failed. No retries permitted until 2026-07-23 00:33:27.777881264 +0000 UTC m=+611.584292493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c5f8583e-370b-4998-a270-4e09f3bb2722-cert") pod "llmisvc-controller-manager-777c6bd7d5-rnpwl" (UID: "c5f8583e-370b-4998-a270-4e09f3bb2722") : secret "llmisvc-webhook-server-cert" not found
Jul 23 00:33:27.301461 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.301424    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chg5l\" (UniqueName: \"kubernetes.io/projected/c5f8583e-370b-4998-a270-4e09f3bb2722-kube-api-access-chg5l\") pod \"llmisvc-controller-manager-777c6bd7d5-rnpwl\" (UID: \"c5f8583e-370b-4998-a270-4e09f3bb2722\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"
Jul 23 00:33:27.682093 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.681997    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/816a95a0-4fa8-48ff-9486-45ac658ee1f2-cert\") pod \"kserve-controller-manager-d6f4b65b8-kh6dp\" (UID: \"816a95a0-4fa8-48ff-9486-45ac658ee1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"
Jul 23 00:33:27.686800 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.686750    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/816a95a0-4fa8-48ff-9486-45ac658ee1f2-cert\") pod \"kserve-controller-manager-d6f4b65b8-kh6dp\" (UID: \"816a95a0-4fa8-48ff-9486-45ac658ee1f2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"
Jul 23 00:33:27.783073 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.783038    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5f8583e-370b-4998-a270-4e09f3bb2722-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rnpwl\" (UID: \"c5f8583e-370b-4998-a270-4e09f3bb2722\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"
Jul 23 00:33:27.785663 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.785640    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c5f8583e-370b-4998-a270-4e09f3bb2722-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rnpwl\" (UID: \"c5f8583e-370b-4998-a270-4e09f3bb2722\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"
Jul 23 00:33:27.937656 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.937613    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"
Jul 23 00:33:27.946124 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:27.946104    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"
Jul 23 00:33:28.177875 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:28.177829    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:33:28.178176 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:28.177923    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:33:28.300071 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:28.300038    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"]
Jul 23 00:33:28.301569 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:33:28.301541    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod816a95a0_4fa8_48ff_9486_45ac658ee1f2.slice/crio-d2b6a4423aa6a521911aad425436af8c6ee27cfbf578ab413b7a438ea1315aac WatchSource:0}: Error finding container d2b6a4423aa6a521911aad425436af8c6ee27cfbf578ab413b7a438ea1315aac: Status 404 returned error can't find the container with id d2b6a4423aa6a521911aad425436af8c6ee27cfbf578ab413b7a438ea1315aac
Jul 23 00:33:28.321952 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:33:28.321900    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc5f8583e_370b_4998_a270_4e09f3bb2722.slice/crio-714bb836e43447625efb03d87940ee6a0e27be9398d94daf8bf2b1f6dfc318f7 WatchSource:0}: Error finding container 714bb836e43447625efb03d87940ee6a0e27be9398d94daf8bf2b1f6dfc318f7: Status 404 returned error can't find the container with id 714bb836e43447625efb03d87940ee6a0e27be9398d94daf8bf2b1f6dfc318f7
Jul 23 00:33:28.323774 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:28.322582    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"]
Jul 23 00:33:28.872914 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:28.872873    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67" event={"ID":"760e166b-3e6c-47c6-b2f6-3ab7b4180f53","Type":"ContainerStarted","Data":"5f372caca96b4ae9669bb0b6ab0b377e5bdf46384bc4c4b21dc8fcdf0d215583"}
Jul 23 00:33:28.873102 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:28.873034    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:28.874752 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:28.874695    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp" event={"ID":"816a95a0-4fa8-48ff-9486-45ac658ee1f2","Type":"ContainerStarted","Data":"d2b6a4423aa6a521911aad425436af8c6ee27cfbf578ab413b7a438ea1315aac"}
Jul 23 00:33:28.876124 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:28.876093    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl" event={"ID":"c5f8583e-370b-4998-a270-4e09f3bb2722","Type":"ContainerStarted","Data":"714bb836e43447625efb03d87940ee6a0e27be9398d94daf8bf2b1f6dfc318f7"}
Jul 23 00:33:28.896843 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:28.896766    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67" podStartSLOduration=1.452845357 podStartE2EDuration="4.896743576s" totalImagesPullingTime="3.443898219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:33:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:33:28.894342251 +0000 UTC m=+612.700753497" watchObservedRunningTime="2026-07-23 00:33:28.896743576 +0000 UTC m=+612.703154810"
Jul 23 00:33:29.883051 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:29.883016    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-g2w67"
Jul 23 00:33:30.887064 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:30.887028    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp" event={"ID":"816a95a0-4fa8-48ff-9486-45ac658ee1f2","Type":"ContainerStarted","Data":"070665a22f02ed4d96ec8702544c67bf874629780917916cc391ba00c0e024f8"}
Jul 23 00:33:30.887555 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:30.887416    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"
Jul 23 00:33:30.906566 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:30.906510    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp" podStartSLOduration=2.440901082 podStartE2EDuration="4.90649324s" totalImagesPullingTime="2.465592158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:33:30.905397409 +0000 UTC m=+614.711808671" watchObservedRunningTime="2026-07-23 00:33:30.90649324 +0000 UTC m=+614.712904472"
Jul 23 00:33:31.892462 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:31.892425    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl" event={"ID":"c5f8583e-370b-4998-a270-4e09f3bb2722","Type":"ContainerStarted","Data":"979b16036c359d2c23497fe336a181144ed2f75bdcfbf329a7f60e2c059e495d"}
Jul 23 00:33:31.892896 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:31.892503    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"
Jul 23 00:33:31.909333 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:31.909253    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl" podStartSLOduration=1.8610040749999999 podStartE2EDuration="4.909235736s" totalImagesPullingTime="3.048231661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:33:31.908745268 +0000 UTC m=+615.715156500" watchObservedRunningTime="2026-07-23 00:33:31.909235736 +0000 UTC m=+615.715646971"
Jul 23 00:33:32.846510 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:33:32.846484    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-qqlgk"
Jul 23 00:34:01.897587 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:01.897557    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kh6dp"
Jul 23 00:34:02.899805 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:02.899774    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rnpwl"
Jul 23 00:34:55.434415 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.434381    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m"]
Jul 23 00:34:55.436482 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.436457    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m"
Jul 23 00:34:55.439075 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.439050    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 23 00:34:55.439211 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.439057    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 00:34:55.439211 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.439155    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-drnx4\""
Jul 23 00:34:55.440169 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.440154    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 00:34:55.452088 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.452065    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m"]
Jul 23 00:34:55.559682 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.559650    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td88p\" (UniqueName: \"kubernetes.io/projected/0fa0eb2d-2929-44ba-8cb3-d5b0bf360756-kube-api-access-td88p\") pod \"dns-operator-controller-manager-864dd78c4c-txt2m\" (UID: \"0fa0eb2d-2929-44ba-8cb3-d5b0bf360756\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m"
Jul 23 00:34:55.660859 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.660824    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td88p\" (UniqueName: \"kubernetes.io/projected/0fa0eb2d-2929-44ba-8cb3-d5b0bf360756-kube-api-access-td88p\") pod \"dns-operator-controller-manager-864dd78c4c-txt2m\" (UID: \"0fa0eb2d-2929-44ba-8cb3-d5b0bf360756\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m"
Jul 23 00:34:55.672203 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.672174    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td88p\" (UniqueName: \"kubernetes.io/projected/0fa0eb2d-2929-44ba-8cb3-d5b0bf360756-kube-api-access-td88p\") pod \"dns-operator-controller-manager-864dd78c4c-txt2m\" (UID: \"0fa0eb2d-2929-44ba-8cb3-d5b0bf360756\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m"
Jul 23 00:34:55.746732 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.746636    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m"
Jul 23 00:34:55.880744 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.880564    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m"]
Jul 23 00:34:55.883904 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:34:55.883875    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fa0eb2d_2929_44ba_8cb3_d5b0bf360756.slice/crio-fc307b77d05674486e32a714c4ff8811598a471f3feace159bcb65ff9bd24029 WatchSource:0}: Error finding container fc307b77d05674486e32a714c4ff8811598a471f3feace159bcb65ff9bd24029: Status 404 returned error can't find the container with id fc307b77d05674486e32a714c4ff8811598a471f3feace159bcb65ff9bd24029
Jul 23 00:34:55.885889 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:55.885872    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 00:34:56.221027 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:56.220995    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m" event={"ID":"0fa0eb2d-2929-44ba-8cb3-d5b0bf360756","Type":"ContainerStarted","Data":"fc307b77d05674486e32a714c4ff8811598a471f3feace159bcb65ff9bd24029"}
Jul 23 00:34:59.237169 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:59.237130    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m" event={"ID":"0fa0eb2d-2929-44ba-8cb3-d5b0bf360756","Type":"ContainerStarted","Data":"a8221dc4a7a952564e73330c60f9829ecf61b790de6616a8fab3877fe9628235"}
Jul 23 00:34:59.237637 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:59.237261    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m"
Jul 23 00:34:59.257760 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:34:59.257667    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m" podStartSLOduration=1.021839654 podStartE2EDuration="4.257651726s" totalImagesPullingTime="3.235812072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:34:59.257227492 +0000 UTC m=+703.063638726" watchObservedRunningTime="2026-07-23 00:34:59.257651726 +0000 UTC m=+703.064062959"
Jul 23 00:35:00.045944 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:00.045912    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"]
Jul 23 00:35:00.048165 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:00.048148    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:00.051186 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:00.051164    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hdqfk\""
Jul 23 00:35:00.067771 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:00.067744    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"]
Jul 23 00:35:00.198547 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:00.198498    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a018e4fc-42f4-4edd-9787-80bf947226a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\" (UID: \"a018e4fc-42f4-4edd-9787-80bf947226a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:00.198547 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:00.198549    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txtmc\" (UniqueName: \"kubernetes.io/projected/a018e4fc-42f4-4edd-9787-80bf947226a6-kube-api-access-txtmc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\" (UID: \"a018e4fc-42f4-4edd-9787-80bf947226a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:00.299417 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:00.299311    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a018e4fc-42f4-4edd-9787-80bf947226a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\" (UID: \"a018e4fc-42f4-4edd-9787-80bf947226a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:00.299417 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:00.299351    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txtmc\" (UniqueName: \"kubernetes.io/projected/a018e4fc-42f4-4edd-9787-80bf947226a6-kube-api-access-txtmc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\" (UID: \"a018e4fc-42f4-4edd-9787-80bf947226a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:00.299822 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:00.299743    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a018e4fc-42f4-4edd-9787-80bf947226a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\" (UID: \"a018e4fc-42f4-4edd-9787-80bf947226a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:00.308010 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:00.307983    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txtmc\" (UniqueName: \"kubernetes.io/projected/a018e4fc-42f4-4edd-9787-80bf947226a6-kube-api-access-txtmc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\" (UID: \"a018e4fc-42f4-4edd-9787-80bf947226a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:00.357997 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:00.357967    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:00.486179 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:00.486148    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"]
Jul 23 00:35:00.490013 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:35:00.489976    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda018e4fc_42f4_4edd_9787_80bf947226a6.slice/crio-103356f8de17cd2722dcd3f86c0b05d6a90cbbd71d8f73454a7e16c244f92d74 WatchSource:0}: Error finding container 103356f8de17cd2722dcd3f86c0b05d6a90cbbd71d8f73454a7e16c244f92d74: Status 404 returned error can't find the container with id 103356f8de17cd2722dcd3f86c0b05d6a90cbbd71d8f73454a7e16c244f92d74
Jul 23 00:35:01.247896 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:01.247864    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9" event={"ID":"a018e4fc-42f4-4edd-9787-80bf947226a6","Type":"ContainerStarted","Data":"103356f8de17cd2722dcd3f86c0b05d6a90cbbd71d8f73454a7e16c244f92d74"}
Jul 23 00:35:08.278069 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:08.278030    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9" event={"ID":"a018e4fc-42f4-4edd-9787-80bf947226a6","Type":"ContainerStarted","Data":"bab28598e466b4a2fa2e281bea651310bcde7575e0140d15505c62ebce48d62e"}
Jul 23 00:35:08.278518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:08.278167    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:08.299508 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:08.299467    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9" podStartSLOduration=1.30614436 podStartE2EDuration="8.299452325s" totalImagesPullingTime="6.993307965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:35:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:35:08.298397911 +0000 UTC m=+712.104809144" watchObservedRunningTime="2026-07-23 00:35:08.299452325 +0000 UTC m=+712.105863558"
Jul 23 00:35:10.243179 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:10.243150    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-txt2m"
Jul 23 00:35:19.284334 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:19.284307    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:21.075769 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.075725    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"]
Jul 23 00:35:21.076264 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.075962    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9" podUID="a018e4fc-42f4-4edd-9787-80bf947226a6" containerName="manager" containerID="cri-o://bab28598e466b4a2fa2e281bea651310bcde7575e0140d15505c62ebce48d62e" gracePeriod=2
Jul 23 00:35:21.086461 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.086431    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"]
Jul 23 00:35:21.097007 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:35:21.096975    2561 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"a018e4fc-42f4-4edd-9787-80bf947226a6\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\": pods \"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:21.103172 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.103149    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"]
Jul 23 00:35:21.103501 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.103484    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a018e4fc-42f4-4edd-9787-80bf947226a6" containerName="manager"
Jul 23 00:35:21.103584 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.103504    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a018e4fc-42f4-4edd-9787-80bf947226a6" containerName="manager"
Jul 23 00:35:21.103643 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.103593    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a018e4fc-42f4-4edd-9787-80bf947226a6" containerName="manager"
Jul 23 00:35:21.105344 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.105324    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"
Jul 23 00:35:21.123534 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.123508    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"]
Jul 23 00:35:21.147644 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:35:21.147604    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\" is forbidden: User \"system:node:ip-10-0-133-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-215.ec2.internal' and this object" podUID="a018e4fc-42f4-4edd-9787-80bf947226a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:21.150969 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.150946    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkrvp\" (UniqueName: \"kubernetes.io/projected/8fad09bf-deb8-4c9d-9c7a-01e57dde442b-kube-api-access-rkrvp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w27r8\" (UID: \"8fad09bf-deb8-4c9d-9c7a-01e57dde442b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"
Jul 23 00:35:21.151096 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.151007    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fad09bf-deb8-4c9d-9c7a-01e57dde442b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w27r8\" (UID: \"8fad09bf-deb8-4c9d-9c7a-01e57dde442b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"
Jul 23 00:35:21.251725 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.251597    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkrvp\" (UniqueName: \"kubernetes.io/projected/8fad09bf-deb8-4c9d-9c7a-01e57dde442b-kube-api-access-rkrvp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w27r8\" (UID: \"8fad09bf-deb8-4c9d-9c7a-01e57dde442b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"
Jul 23 00:35:21.251725 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.251686    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fad09bf-deb8-4c9d-9c7a-01e57dde442b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w27r8\" (UID: \"8fad09bf-deb8-4c9d-9c7a-01e57dde442b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"
Jul 23 00:35:21.252113 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.252084    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fad09bf-deb8-4c9d-9c7a-01e57dde442b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w27r8\" (UID: \"8fad09bf-deb8-4c9d-9c7a-01e57dde442b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"
Jul 23 00:35:21.261639 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.261610    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkrvp\" (UniqueName: \"kubernetes.io/projected/8fad09bf-deb8-4c9d-9c7a-01e57dde442b-kube-api-access-rkrvp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w27r8\" (UID: \"8fad09bf-deb8-4c9d-9c7a-01e57dde442b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"
Jul 23 00:35:21.314192 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.314171    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:21.316563 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:35:21.316540    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\" is forbidden: User \"system:node:ip-10-0-133-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-215.ec2.internal' and this object" podUID="a018e4fc-42f4-4edd-9787-80bf947226a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:21.332278 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.332218    2561 generic.go:356] "Generic (PLEG): container finished" podID="a018e4fc-42f4-4edd-9787-80bf947226a6" containerID="bab28598e466b4a2fa2e281bea651310bcde7575e0140d15505c62ebce48d62e" exitCode=0
Jul 23 00:35:21.332278 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.332269    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:21.332449 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.332328    2561 scope.go:117] "RemoveContainer" containerID="bab28598e466b4a2fa2e281bea651310bcde7575e0140d15505c62ebce48d62e"
Jul 23 00:35:21.334615 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:35:21.334587    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\" is forbidden: User \"system:node:ip-10-0-133-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-215.ec2.internal' and this object" podUID="a018e4fc-42f4-4edd-9787-80bf947226a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:21.340416 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.340393    2561 scope.go:117] "RemoveContainer" containerID="bab28598e466b4a2fa2e281bea651310bcde7575e0140d15505c62ebce48d62e"
Jul 23 00:35:21.340671 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:35:21.340650    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bab28598e466b4a2fa2e281bea651310bcde7575e0140d15505c62ebce48d62e\": container with ID starting with bab28598e466b4a2fa2e281bea651310bcde7575e0140d15505c62ebce48d62e not found: ID does not exist" containerID="bab28598e466b4a2fa2e281bea651310bcde7575e0140d15505c62ebce48d62e"
Jul 23 00:35:21.340729 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.340680    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bab28598e466b4a2fa2e281bea651310bcde7575e0140d15505c62ebce48d62e"} err="failed to get container status \"bab28598e466b4a2fa2e281bea651310bcde7575e0140d15505c62ebce48d62e\": rpc error: code = NotFound desc = could not find container \"bab28598e466b4a2fa2e281bea651310bcde7575e0140d15505c62ebce48d62e\": container with ID starting with bab28598e466b4a2fa2e281bea651310bcde7575e0140d15505c62ebce48d62e not found: ID does not exist"
Jul 23 00:35:21.352974 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.352957    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a018e4fc-42f4-4edd-9787-80bf947226a6-extensions-socket-volume\") pod \"a018e4fc-42f4-4edd-9787-80bf947226a6\" (UID: \"a018e4fc-42f4-4edd-9787-80bf947226a6\") "
Jul 23 00:35:21.353047 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.352984    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txtmc\" (UniqueName: \"kubernetes.io/projected/a018e4fc-42f4-4edd-9787-80bf947226a6-kube-api-access-txtmc\") pod \"a018e4fc-42f4-4edd-9787-80bf947226a6\" (UID: \"a018e4fc-42f4-4edd-9787-80bf947226a6\") "
Jul 23 00:35:21.353314 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.353295    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a018e4fc-42f4-4edd-9787-80bf947226a6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a018e4fc-42f4-4edd-9787-80bf947226a6" (UID: "a018e4fc-42f4-4edd-9787-80bf947226a6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:35:21.355049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.355024    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a018e4fc-42f4-4edd-9787-80bf947226a6-kube-api-access-txtmc" (OuterVolumeSpecName: "kube-api-access-txtmc") pod "a018e4fc-42f4-4edd-9787-80bf947226a6" (UID: "a018e4fc-42f4-4edd-9787-80bf947226a6"). InnerVolumeSpecName "kube-api-access-txtmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:35:21.453575 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.453546    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a018e4fc-42f4-4edd-9787-80bf947226a6-extensions-socket-volume\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:35:21.453575 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.453571    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txtmc\" (UniqueName: \"kubernetes.io/projected/a018e4fc-42f4-4edd-9787-80bf947226a6-kube-api-access-txtmc\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:35:21.459498 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.459477    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"
Jul 23 00:35:21.591432 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:21.591337    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"]
Jul 23 00:35:21.593540 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:35:21.593520    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fad09bf_deb8_4c9d_9c7a_01e57dde442b.slice/crio-cdd18ea57202616c7765e84a64b45fe405fe7a97f68f6e162a0a81e3e0f6f17c WatchSource:0}: Error finding container cdd18ea57202616c7765e84a64b45fe405fe7a97f68f6e162a0a81e3e0f6f17c: Status 404 returned error can't find the container with id cdd18ea57202616c7765e84a64b45fe405fe7a97f68f6e162a0a81e3e0f6f17c
Jul 23 00:35:21.651333 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:35:21.651307    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\" is forbidden: User \"system:node:ip-10-0-133-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-215.ec2.internal' and this object" podUID="a018e4fc-42f4-4edd-9787-80bf947226a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:22.339442 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:22.339400    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8" event={"ID":"8fad09bf-deb8-4c9d-9c7a-01e57dde442b","Type":"ContainerStarted","Data":"ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc"}
Jul 23 00:35:22.339841 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:22.339490    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8" event={"ID":"8fad09bf-deb8-4c9d-9c7a-01e57dde442b","Type":"ContainerStarted","Data":"cdd18ea57202616c7765e84a64b45fe405fe7a97f68f6e162a0a81e3e0f6f17c"}
Jul 23 00:35:22.339841 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:22.339513    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"
Jul 23 00:35:22.372916 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:22.372873    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8" podStartSLOduration=1.372860967 podStartE2EDuration="1.372860967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:35:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:35:22.372690224 +0000 UTC m=+726.179101457" watchObservedRunningTime="2026-07-23 00:35:22.372860967 +0000 UTC m=+726.179272199"
Jul 23 00:35:22.375031 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:35:22.375005    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-ng5k9\" is forbidden: User \"system:node:ip-10-0-133-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-215.ec2.internal' and this object" podUID="a018e4fc-42f4-4edd-9787-80bf947226a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ng5k9"
Jul 23 00:35:22.787272 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:22.787237    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a018e4fc-42f4-4edd-9787-80bf947226a6" path="/var/lib/kubelet/pods/a018e4fc-42f4-4edd-9787-80bf947226a6/volumes"
Jul 23 00:35:33.345449 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:33.345416    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"
Jul 23 00:35:49.529713 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.529676    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"]
Jul 23 00:35:49.530277 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.529905    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8" podUID="8fad09bf-deb8-4c9d-9c7a-01e57dde442b" containerName="manager" containerID="cri-o://ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc" gracePeriod=10
Jul 23 00:35:49.767481 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.767452    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"]
Jul 23 00:35:49.770283 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.770262    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"
Jul 23 00:35:49.774564 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.774546    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"
Jul 23 00:35:49.780950 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.780904    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"]
Jul 23 00:35:49.873078 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.873044    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fad09bf-deb8-4c9d-9c7a-01e57dde442b-extensions-socket-volume\") pod \"8fad09bf-deb8-4c9d-9c7a-01e57dde442b\" (UID: \"8fad09bf-deb8-4c9d-9c7a-01e57dde442b\") "
Jul 23 00:35:49.873237 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.873116    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkrvp\" (UniqueName: \"kubernetes.io/projected/8fad09bf-deb8-4c9d-9c7a-01e57dde442b-kube-api-access-rkrvp\") pod \"8fad09bf-deb8-4c9d-9c7a-01e57dde442b\" (UID: \"8fad09bf-deb8-4c9d-9c7a-01e57dde442b\") "
Jul 23 00:35:49.873344 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.873310    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nvnc\" (UniqueName: \"kubernetes.io/projected/9033bade-2184-4fd4-84ce-db1f703471d9-kube-api-access-2nvnc\") pod \"kuadrant-operator-controller-manager-86878c769f-l6fjx\" (UID: \"9033bade-2184-4fd4-84ce-db1f703471d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"
Jul 23 00:35:49.873487 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.873467    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9033bade-2184-4fd4-84ce-db1f703471d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l6fjx\" (UID: \"9033bade-2184-4fd4-84ce-db1f703471d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"
Jul 23 00:35:49.873534 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.873491    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fad09bf-deb8-4c9d-9c7a-01e57dde442b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8fad09bf-deb8-4c9d-9c7a-01e57dde442b" (UID: "8fad09bf-deb8-4c9d-9c7a-01e57dde442b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:35:49.875236 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.875219    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fad09bf-deb8-4c9d-9c7a-01e57dde442b-kube-api-access-rkrvp" (OuterVolumeSpecName: "kube-api-access-rkrvp") pod "8fad09bf-deb8-4c9d-9c7a-01e57dde442b" (UID: "8fad09bf-deb8-4c9d-9c7a-01e57dde442b"). InnerVolumeSpecName "kube-api-access-rkrvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:35:49.974730 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.974696    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nvnc\" (UniqueName: \"kubernetes.io/projected/9033bade-2184-4fd4-84ce-db1f703471d9-kube-api-access-2nvnc\") pod \"kuadrant-operator-controller-manager-86878c769f-l6fjx\" (UID: \"9033bade-2184-4fd4-84ce-db1f703471d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"
Jul 23 00:35:49.974895 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.974771    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9033bade-2184-4fd4-84ce-db1f703471d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l6fjx\" (UID: \"9033bade-2184-4fd4-84ce-db1f703471d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"
Jul 23 00:35:49.974895 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.974831    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkrvp\" (UniqueName: \"kubernetes.io/projected/8fad09bf-deb8-4c9d-9c7a-01e57dde442b-kube-api-access-rkrvp\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:35:49.974895 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.974847    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fad09bf-deb8-4c9d-9c7a-01e57dde442b-extensions-socket-volume\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:35:49.975180 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.975160    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9033bade-2184-4fd4-84ce-db1f703471d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l6fjx\" (UID: \"9033bade-2184-4fd4-84ce-db1f703471d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"
Jul 23 00:35:49.983151 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:49.983119    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nvnc\" (UniqueName: \"kubernetes.io/projected/9033bade-2184-4fd4-84ce-db1f703471d9-kube-api-access-2nvnc\") pod \"kuadrant-operator-controller-manager-86878c769f-l6fjx\" (UID: \"9033bade-2184-4fd4-84ce-db1f703471d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"
Jul 23 00:35:50.081512 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.081429    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"
Jul 23 00:35:50.208046 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.208016    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"]
Jul 23 00:35:50.210690 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:35:50.210662    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9033bade_2184_4fd4_84ce_db1f703471d9.slice/crio-3d45af9db75ded04dcb71e8666cb5cd402f61949cdefe79b8c0858b9d8b6f582 WatchSource:0}: Error finding container 3d45af9db75ded04dcb71e8666cb5cd402f61949cdefe79b8c0858b9d8b6f582: Status 404 returned error can't find the container with id 3d45af9db75ded04dcb71e8666cb5cd402f61949cdefe79b8c0858b9d8b6f582
Jul 23 00:35:50.459657 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.459624    2561 generic.go:356] "Generic (PLEG): container finished" podID="8fad09bf-deb8-4c9d-9c7a-01e57dde442b" containerID="ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc" exitCode=0
Jul 23 00:35:50.459831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.459670    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"
Jul 23 00:35:50.459831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.459714    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8" event={"ID":"8fad09bf-deb8-4c9d-9c7a-01e57dde442b","Type":"ContainerDied","Data":"ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc"}
Jul 23 00:35:50.459831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.459768    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8" event={"ID":"8fad09bf-deb8-4c9d-9c7a-01e57dde442b","Type":"ContainerDied","Data":"cdd18ea57202616c7765e84a64b45fe405fe7a97f68f6e162a0a81e3e0f6f17c"}
Jul 23 00:35:50.459831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.459795    2561 scope.go:117] "RemoveContainer" containerID="ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc"
Jul 23 00:35:50.461505 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.461480    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx" event={"ID":"9033bade-2184-4fd4-84ce-db1f703471d9","Type":"ContainerStarted","Data":"ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5"}
Jul 23 00:35:50.461630 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.461519    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx" event={"ID":"9033bade-2184-4fd4-84ce-db1f703471d9","Type":"ContainerStarted","Data":"3d45af9db75ded04dcb71e8666cb5cd402f61949cdefe79b8c0858b9d8b6f582"}
Jul 23 00:35:50.461690 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.461630    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"
Jul 23 00:35:50.468999 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.468971    2561 scope.go:117] "RemoveContainer" containerID="ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc"
Jul 23 00:35:50.469253 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:35:50.469233    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc\": container with ID starting with ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc not found: ID does not exist" containerID="ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc"
Jul 23 00:35:50.469351 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.469259    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc"} err="failed to get container status \"ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc\": rpc error: code = NotFound desc = could not find container \"ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc\": container with ID starting with ea5d8700ac0eabda875f87af2ebeaffecb80d6c02b8839c89d66171f12d7d9dc not found: ID does not exist"
Jul 23 00:35:50.482777 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.482729    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx" podStartSLOduration=1.482715638 podStartE2EDuration="1.482715638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:35:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:35:50.481483042 +0000 UTC m=+754.287894275" watchObservedRunningTime="2026-07-23 00:35:50.482715638 +0000 UTC m=+754.289126870"
Jul 23 00:35:50.495895 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.495870    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"]
Jul 23 00:35:50.500454 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.500431    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w27r8"]
Jul 23 00:35:50.786210 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:35:50.786132    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fad09bf-deb8-4c9d-9c7a-01e57dde442b" path="/var/lib/kubelet/pods/8fad09bf-deb8-4c9d-9c7a-01e57dde442b/volumes"
Jul 23 00:36:01.470137 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:01.470111    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"
Jul 23 00:36:10.196826 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.196789    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"]
Jul 23 00:36:10.197328 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.197236    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fad09bf-deb8-4c9d-9c7a-01e57dde442b" containerName="manager"
Jul 23 00:36:10.197328 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.197253    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fad09bf-deb8-4c9d-9c7a-01e57dde442b" containerName="manager"
Jul 23 00:36:10.197492 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.197348    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fad09bf-deb8-4c9d-9c7a-01e57dde442b" containerName="manager"
Jul 23 00:36:10.199861 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.199840    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"
Jul 23 00:36:10.202445 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.202421    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nzwvk\""
Jul 23 00:36:10.202559 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.202445    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 23 00:36:10.207798 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.207772    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"]
Jul 23 00:36:10.294254 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.294219    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"]
Jul 23 00:36:10.342072 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.342036    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52-config-file\") pod \"limitador-limitador-64c94fd64f-ps7x4\" (UID: \"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"
Jul 23 00:36:10.342253 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.342097    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7crjs\" (UniqueName: \"kubernetes.io/projected/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52-kube-api-access-7crjs\") pod \"limitador-limitador-64c94fd64f-ps7x4\" (UID: \"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"
Jul 23 00:36:10.443112 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.443074    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7crjs\" (UniqueName: \"kubernetes.io/projected/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52-kube-api-access-7crjs\") pod \"limitador-limitador-64c94fd64f-ps7x4\" (UID: \"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"
Jul 23 00:36:10.443307 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.443172    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52-config-file\") pod \"limitador-limitador-64c94fd64f-ps7x4\" (UID: \"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"
Jul 23 00:36:10.443817 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.443793    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52-config-file\") pod \"limitador-limitador-64c94fd64f-ps7x4\" (UID: \"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"
Jul 23 00:36:10.451184 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.451131    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7crjs\" (UniqueName: \"kubernetes.io/projected/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52-kube-api-access-7crjs\") pod \"limitador-limitador-64c94fd64f-ps7x4\" (UID: \"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"
Jul 23 00:36:10.511032 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.510997    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"
Jul 23 00:36:10.640240 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:10.640206    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"]
Jul 23 00:36:10.643384 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:36:10.643337    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39cbb9b3_0cc8_41a9_bfc5_364a1e151b52.slice/crio-f805b00bd712b98d31ceae32598973e151778c3066bc90acd0995690e6339edd WatchSource:0}: Error finding container f805b00bd712b98d31ceae32598973e151778c3066bc90acd0995690e6339edd: Status 404 returned error can't find the container with id f805b00bd712b98d31ceae32598973e151778c3066bc90acd0995690e6339edd
Jul 23 00:36:11.553044 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:11.553005    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4" event={"ID":"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52","Type":"ContainerStarted","Data":"f805b00bd712b98d31ceae32598973e151778c3066bc90acd0995690e6339edd"}
Jul 23 00:36:13.566939 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:13.566843    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4" event={"ID":"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52","Type":"ContainerStarted","Data":"6759501fdbfd6ba3698fcb1db4edc2720acc9e90f66ed743018cdb813b6dbe31"}
Jul 23 00:36:13.567318 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:13.566979    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"
Jul 23 00:36:13.586794 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:13.586750    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4" podStartSLOduration=1.045670199 podStartE2EDuration="3.58673563s" totalImagesPullingTime="2.541065431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:36:13.584341822 +0000 UTC m=+777.390753056" watchObservedRunningTime="2026-07-23 00:36:13.58673563 +0000 UTC m=+777.393146863"
Jul 23 00:36:24.572018 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:24.571986    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"
Jul 23 00:36:26.715657 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:26.715626    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"]
Jul 23 00:36:26.716056 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:26.715796    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4" podUID="39cbb9b3-0cc8-41a9-bfc5-364a1e151b52" containerName="limitador" containerID="cri-o://6759501fdbfd6ba3698fcb1db4edc2720acc9e90f66ed743018cdb813b6dbe31" gracePeriod=30
Jul 23 00:36:27.629078 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:27.629042    2561 generic.go:356] "Generic (PLEG): container finished" podID="39cbb9b3-0cc8-41a9-bfc5-364a1e151b52" containerID="6759501fdbfd6ba3698fcb1db4edc2720acc9e90f66ed743018cdb813b6dbe31" exitCode=0
Jul 23 00:36:27.629243 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:27.629118    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4" event={"ID":"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52","Type":"ContainerDied","Data":"6759501fdbfd6ba3698fcb1db4edc2720acc9e90f66ed743018cdb813b6dbe31"}
Jul 23 00:36:27.655672 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:27.655642    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"
Jul 23 00:36:27.675418 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:27.675394    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52-config-file\") pod \"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52\" (UID: \"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52\") "
Jul 23 00:36:27.675547 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:27.675433    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7crjs\" (UniqueName: \"kubernetes.io/projected/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52-kube-api-access-7crjs\") pod \"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52\" (UID: \"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52\") "
Jul 23 00:36:27.675770 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:27.675746    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52-config-file" (OuterVolumeSpecName: "config-file") pod "39cbb9b3-0cc8-41a9-bfc5-364a1e151b52" (UID: "39cbb9b3-0cc8-41a9-bfc5-364a1e151b52"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:36:27.677624 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:27.677593    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52-kube-api-access-7crjs" (OuterVolumeSpecName: "kube-api-access-7crjs") pod "39cbb9b3-0cc8-41a9-bfc5-364a1e151b52" (UID: "39cbb9b3-0cc8-41a9-bfc5-364a1e151b52"). InnerVolumeSpecName "kube-api-access-7crjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:36:27.776183 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:27.776147    2561 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52-config-file\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:36:27.776183 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:27.776176    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7crjs\" (UniqueName: \"kubernetes.io/projected/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52-kube-api-access-7crjs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:36:28.635865 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:28.635836    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"
Jul 23 00:36:28.635865 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:28.635854    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ps7x4" event={"ID":"39cbb9b3-0cc8-41a9-bfc5-364a1e151b52","Type":"ContainerDied","Data":"f805b00bd712b98d31ceae32598973e151778c3066bc90acd0995690e6339edd"}
Jul 23 00:36:28.636076 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:28.635909    2561 scope.go:117] "RemoveContainer" containerID="6759501fdbfd6ba3698fcb1db4edc2720acc9e90f66ed743018cdb813b6dbe31"
Jul 23 00:36:28.661001 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:28.660973    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"]
Jul 23 00:36:28.667405 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:28.667382    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ps7x4"]
Jul 23 00:36:28.787375 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:28.787328    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39cbb9b3-0cc8-41a9-bfc5-364a1e151b52" path="/var/lib/kubelet/pods/39cbb9b3-0cc8-41a9-bfc5-364a1e151b52/volumes"
Jul 23 00:36:29.076708 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.076675    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"]
Jul 23 00:36:29.077124 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.077110    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cbb9b3-0cc8-41a9-bfc5-364a1e151b52" containerName="limitador"
Jul 23 00:36:29.077168 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.077129    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cbb9b3-0cc8-41a9-bfc5-364a1e151b52" containerName="limitador"
Jul 23 00:36:29.077219 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.077209    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cbb9b3-0cc8-41a9-bfc5-364a1e151b52" containerName="limitador"
Jul 23 00:36:29.080131 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.080110    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"
Jul 23 00:36:29.082568 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.082541    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 00:36:29.082696 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.082670    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 23 00:36:29.082906 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.082887    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 00:36:29.083667 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.083652    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-45b2k\""
Jul 23 00:36:29.088874 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.088853    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"]
Jul 23 00:36:29.185076 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.185045    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czcmp\" (UniqueName: \"kubernetes.io/projected/792026ff-c2fc-46f0-abd9-ead17efcde54-kube-api-access-czcmp\") pod \"postgres-69fb4594cf-2cb44\" (UID: \"792026ff-c2fc-46f0-abd9-ead17efcde54\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"
Jul 23 00:36:29.185237 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.185088    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/792026ff-c2fc-46f0-abd9-ead17efcde54-data\") pod \"postgres-69fb4594cf-2cb44\" (UID: \"792026ff-c2fc-46f0-abd9-ead17efcde54\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"
Jul 23 00:36:29.286290 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.286257    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/792026ff-c2fc-46f0-abd9-ead17efcde54-data\") pod \"postgres-69fb4594cf-2cb44\" (UID: \"792026ff-c2fc-46f0-abd9-ead17efcde54\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"
Jul 23 00:36:29.286502 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.286327    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czcmp\" (UniqueName: \"kubernetes.io/projected/792026ff-c2fc-46f0-abd9-ead17efcde54-kube-api-access-czcmp\") pod \"postgres-69fb4594cf-2cb44\" (UID: \"792026ff-c2fc-46f0-abd9-ead17efcde54\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"
Jul 23 00:36:29.286691 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.286673    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/792026ff-c2fc-46f0-abd9-ead17efcde54-data\") pod \"postgres-69fb4594cf-2cb44\" (UID: \"792026ff-c2fc-46f0-abd9-ead17efcde54\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"
Jul 23 00:36:29.294544 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.294513    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czcmp\" (UniqueName: \"kubernetes.io/projected/792026ff-c2fc-46f0-abd9-ead17efcde54-kube-api-access-czcmp\") pod \"postgres-69fb4594cf-2cb44\" (UID: \"792026ff-c2fc-46f0-abd9-ead17efcde54\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"
Jul 23 00:36:29.392826 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.392749    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"
Jul 23 00:36:29.516494 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.516474    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"]
Jul 23 00:36:29.518670 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:36:29.518642    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod792026ff_c2fc_46f0_abd9_ead17efcde54.slice/crio-b19d623f55784a12a29eaab9462f6017aac78eb67ce9bb501b261425e480a09e WatchSource:0}: Error finding container b19d623f55784a12a29eaab9462f6017aac78eb67ce9bb501b261425e480a09e: Status 404 returned error can't find the container with id b19d623f55784a12a29eaab9462f6017aac78eb67ce9bb501b261425e480a09e
Jul 23 00:36:29.641693 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:29.641662    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44" event={"ID":"792026ff-c2fc-46f0-abd9-ead17efcde54","Type":"ContainerStarted","Data":"b19d623f55784a12a29eaab9462f6017aac78eb67ce9bb501b261425e480a09e"}
Jul 23 00:36:34.669936 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:34.669898    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44" event={"ID":"792026ff-c2fc-46f0-abd9-ead17efcde54","Type":"ContainerStarted","Data":"dcd233af5270000267b1565148913b862969dfe1d13ea3ad47c104ac4d7ea491"}
Jul 23 00:36:34.670330 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:34.669997    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"
Jul 23 00:36:34.685038 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:34.684981    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44" podStartSLOduration=0.779023307 podStartE2EDuration="5.684963942s" totalImagesPullingTime="4.905940635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:36:34.684289301 +0000 UTC m=+798.490700535" watchObservedRunningTime="2026-07-23 00:36:34.684963942 +0000 UTC m=+798.491375176"
Jul 23 00:36:40.707592 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:40.707515    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2cb44"
Jul 23 00:36:47.380821 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.380785    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74c5d79b89-tfw68"]
Jul 23 00:36:47.383999 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.383980    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:36:47.386539 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.386521    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kzpcs\""
Jul 23 00:36:47.386641 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.386571    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 00:36:47.387421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.387407    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 00:36:47.394518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.394472    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74c5d79b89-tfw68"]
Jul 23 00:36:47.443552 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.443525    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-789ww\" (UniqueName: \"kubernetes.io/projected/64af7f56-51c6-4256-9957-cb2e84801629-kube-api-access-789ww\") pod \"maas-controller-74c5d79b89-tfw68\" (UID: \"64af7f56-51c6-4256-9957-cb2e84801629\") " pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:36:47.443724 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.443591    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64af7f56-51c6-4256-9957-cb2e84801629-webhook-cert\") pod \"maas-controller-74c5d79b89-tfw68\" (UID: \"64af7f56-51c6-4256-9957-cb2e84801629\") " pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:36:47.544849 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.544819    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64af7f56-51c6-4256-9957-cb2e84801629-webhook-cert\") pod \"maas-controller-74c5d79b89-tfw68\" (UID: \"64af7f56-51c6-4256-9957-cb2e84801629\") " pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:36:47.545062 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.544869    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-789ww\" (UniqueName: \"kubernetes.io/projected/64af7f56-51c6-4256-9957-cb2e84801629-kube-api-access-789ww\") pod \"maas-controller-74c5d79b89-tfw68\" (UID: \"64af7f56-51c6-4256-9957-cb2e84801629\") " pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:36:47.545062 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:36:47.545006    2561 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 23 00:36:47.545179 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:36:47.545108    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64af7f56-51c6-4256-9957-cb2e84801629-webhook-cert podName:64af7f56-51c6-4256-9957-cb2e84801629 nodeName:}" failed. No retries permitted until 2026-07-23 00:36:48.045084224 +0000 UTC m=+811.851495443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/64af7f56-51c6-4256-9957-cb2e84801629-webhook-cert") pod "maas-controller-74c5d79b89-tfw68" (UID: "64af7f56-51c6-4256-9957-cb2e84801629") : secret "maas-controller-webhook-cert" not found
Jul 23 00:36:47.549467 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.549444    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-795c797fc6-54kwp"]
Jul 23 00:36:47.552551 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.552533    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:36:47.555389 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.555355    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-789ww\" (UniqueName: \"kubernetes.io/projected/64af7f56-51c6-4256-9957-cb2e84801629-kube-api-access-789ww\") pod \"maas-controller-74c5d79b89-tfw68\" (UID: \"64af7f56-51c6-4256-9957-cb2e84801629\") " pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:36:47.563496 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.563473    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-795c797fc6-54kwp"]
Jul 23 00:36:47.645568 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.645471    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj8m9\" (UniqueName: \"kubernetes.io/projected/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-kube-api-access-rj8m9\") pod \"maas-controller-795c797fc6-54kwp\" (UID: \"5d39aa6e-fbd2-48c8-b7ba-8b614abce076\") " pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:36:47.645568 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.645540    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-webhook-cert\") pod \"maas-controller-795c797fc6-54kwp\" (UID: \"5d39aa6e-fbd2-48c8-b7ba-8b614abce076\") " pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:36:47.745990 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.745957    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj8m9\" (UniqueName: \"kubernetes.io/projected/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-kube-api-access-rj8m9\") pod \"maas-controller-795c797fc6-54kwp\" (UID: \"5d39aa6e-fbd2-48c8-b7ba-8b614abce076\") " pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:36:47.746201 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.746012    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-webhook-cert\") pod \"maas-controller-795c797fc6-54kwp\" (UID: \"5d39aa6e-fbd2-48c8-b7ba-8b614abce076\") " pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:36:47.746201 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:36:47.746108    2561 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 23 00:36:47.746201 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:36:47.746155    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-webhook-cert podName:5d39aa6e-fbd2-48c8-b7ba-8b614abce076 nodeName:}" failed. No retries permitted until 2026-07-23 00:36:48.246142527 +0000 UTC m=+812.052553740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-webhook-cert") pod "maas-controller-795c797fc6-54kwp" (UID: "5d39aa6e-fbd2-48c8-b7ba-8b614abce076") : secret "maas-controller-webhook-cert" not found
Jul 23 00:36:47.756929 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:47.756898    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj8m9\" (UniqueName: \"kubernetes.io/projected/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-kube-api-access-rj8m9\") pod \"maas-controller-795c797fc6-54kwp\" (UID: \"5d39aa6e-fbd2-48c8-b7ba-8b614abce076\") " pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:36:48.048336 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:48.048298    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64af7f56-51c6-4256-9957-cb2e84801629-webhook-cert\") pod \"maas-controller-74c5d79b89-tfw68\" (UID: \"64af7f56-51c6-4256-9957-cb2e84801629\") " pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:36:48.050763 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:48.050743    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64af7f56-51c6-4256-9957-cb2e84801629-webhook-cert\") pod \"maas-controller-74c5d79b89-tfw68\" (UID: \"64af7f56-51c6-4256-9957-cb2e84801629\") " pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:36:48.249656 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:48.249625    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-webhook-cert\") pod \"maas-controller-795c797fc6-54kwp\" (UID: \"5d39aa6e-fbd2-48c8-b7ba-8b614abce076\") " pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:36:48.252001 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:48.251969    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-webhook-cert\") pod \"maas-controller-795c797fc6-54kwp\" (UID: \"5d39aa6e-fbd2-48c8-b7ba-8b614abce076\") " pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:36:48.294923 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:48.294889    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:36:48.421061 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:48.421035    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74c5d79b89-tfw68"]
Jul 23 00:36:48.422616 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:36:48.422592    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64af7f56_51c6_4256_9957_cb2e84801629.slice/crio-d9684efaced913501636f7eafb71b2b5351b82a74a2c3fbb095d60de4ff557aa WatchSource:0}: Error finding container d9684efaced913501636f7eafb71b2b5351b82a74a2c3fbb095d60de4ff557aa: Status 404 returned error can't find the container with id d9684efaced913501636f7eafb71b2b5351b82a74a2c3fbb095d60de4ff557aa
Jul 23 00:36:48.470615 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:48.470576    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:36:48.595894 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:48.595873    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-795c797fc6-54kwp"]
Jul 23 00:36:48.597613 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:36:48.597591    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d39aa6e_fbd2_48c8_b7ba_8b614abce076.slice/crio-32b5cb9fff6ec1f2171299b978b3cbfbe8a8135ce4cf9a02ffd80eec6c86e8f6 WatchSource:0}: Error finding container 32b5cb9fff6ec1f2171299b978b3cbfbe8a8135ce4cf9a02ffd80eec6c86e8f6: Status 404 returned error can't find the container with id 32b5cb9fff6ec1f2171299b978b3cbfbe8a8135ce4cf9a02ffd80eec6c86e8f6
Jul 23 00:36:48.732944 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:48.732908    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795c797fc6-54kwp" event={"ID":"5d39aa6e-fbd2-48c8-b7ba-8b614abce076","Type":"ContainerStarted","Data":"32b5cb9fff6ec1f2171299b978b3cbfbe8a8135ce4cf9a02ffd80eec6c86e8f6"}
Jul 23 00:36:48.733972 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:48.733946    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74c5d79b89-tfw68" event={"ID":"64af7f56-51c6-4256-9957-cb2e84801629","Type":"ContainerStarted","Data":"d9684efaced913501636f7eafb71b2b5351b82a74a2c3fbb095d60de4ff557aa"}
Jul 23 00:36:51.753358 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:51.753315    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795c797fc6-54kwp" event={"ID":"5d39aa6e-fbd2-48c8-b7ba-8b614abce076","Type":"ContainerStarted","Data":"946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c"}
Jul 23 00:36:51.753856 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:51.753405    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:36:51.754825 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:51.754795    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74c5d79b89-tfw68" event={"ID":"64af7f56-51c6-4256-9957-cb2e84801629","Type":"ContainerStarted","Data":"32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb"}
Jul 23 00:36:51.754959 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:51.754872    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:36:51.775919 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:51.775870    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-795c797fc6-54kwp" podStartSLOduration=2.393376592 podStartE2EDuration="4.77585294s" totalImagesPullingTime="2.382476348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:36:51.774637207 +0000 UTC m=+815.581056385" watchObservedRunningTime="2026-07-23 00:36:51.77585294 +0000 UTC m=+815.582264172"
Jul 23 00:36:51.796489 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:36:51.796447    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74c5d79b89-tfw68" podStartSLOduration=2.244925357 podStartE2EDuration="4.796430052s" totalImagesPullingTime="2.551504695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:36:51.794500051 +0000 UTC m=+815.600911297" watchObservedRunningTime="2026-07-23 00:36:51.796430052 +0000 UTC m=+815.602841304"
Jul 23 00:37:02.761768 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:02.761738    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:37:02.762175 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:02.761809    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:37:02.820345 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:02.820313    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74c5d79b89-tfw68"]
Jul 23 00:37:02.820563 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:02.820515    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74c5d79b89-tfw68" podUID="64af7f56-51c6-4256-9957-cb2e84801629" containerName="manager" containerID="cri-o://32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb" gracePeriod=30
Jul 23 00:37:03.058981 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.058957    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:37:03.174132 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.174104    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-789ww\" (UniqueName: \"kubernetes.io/projected/64af7f56-51c6-4256-9957-cb2e84801629-kube-api-access-789ww\") pod \"64af7f56-51c6-4256-9957-cb2e84801629\" (UID: \"64af7f56-51c6-4256-9957-cb2e84801629\") "
Jul 23 00:37:03.174332 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.174152    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64af7f56-51c6-4256-9957-cb2e84801629-webhook-cert\") pod \"64af7f56-51c6-4256-9957-cb2e84801629\" (UID: \"64af7f56-51c6-4256-9957-cb2e84801629\") "
Jul 23 00:37:03.176351 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.176322    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64af7f56-51c6-4256-9957-cb2e84801629-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "64af7f56-51c6-4256-9957-cb2e84801629" (UID: "64af7f56-51c6-4256-9957-cb2e84801629"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:37:03.176351 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.176324    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64af7f56-51c6-4256-9957-cb2e84801629-kube-api-access-789ww" (OuterVolumeSpecName: "kube-api-access-789ww") pod "64af7f56-51c6-4256-9957-cb2e84801629" (UID: "64af7f56-51c6-4256-9957-cb2e84801629"). InnerVolumeSpecName "kube-api-access-789ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:37:03.274982 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.274945    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-789ww\" (UniqueName: \"kubernetes.io/projected/64af7f56-51c6-4256-9957-cb2e84801629-kube-api-access-789ww\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:37:03.274982 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.274977    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/64af7f56-51c6-4256-9957-cb2e84801629-webhook-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:37:03.805518 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.805485    2561 generic.go:356] "Generic (PLEG): container finished" podID="64af7f56-51c6-4256-9957-cb2e84801629" containerID="32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb" exitCode=0
Jul 23 00:37:03.805949 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.805528    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74c5d79b89-tfw68"
Jul 23 00:37:03.805949 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.805569    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74c5d79b89-tfw68" event={"ID":"64af7f56-51c6-4256-9957-cb2e84801629","Type":"ContainerDied","Data":"32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb"}
Jul 23 00:37:03.805949 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.805615    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74c5d79b89-tfw68" event={"ID":"64af7f56-51c6-4256-9957-cb2e84801629","Type":"ContainerDied","Data":"d9684efaced913501636f7eafb71b2b5351b82a74a2c3fbb095d60de4ff557aa"}
Jul 23 00:37:03.805949 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.805637    2561 scope.go:117] "RemoveContainer" containerID="32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb"
Jul 23 00:37:03.814795 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.814779    2561 scope.go:117] "RemoveContainer" containerID="32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb"
Jul 23 00:37:03.815081 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:37:03.815064    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb\": container with ID starting with 32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb not found: ID does not exist" containerID="32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb"
Jul 23 00:37:03.815128 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.815090    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb"} err="failed to get container status \"32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb\": rpc error: code = NotFound desc = could not find container \"32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb\": container with ID starting with 32e467db06fa7ee28d7e9ac4ab53c926319c923604f70b3d75bf84047ce4bffb not found: ID does not exist"
Jul 23 00:37:03.829189 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.829165    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74c5d79b89-tfw68"]
Jul 23 00:37:03.833134 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:03.833109    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74c5d79b89-tfw68"]
Jul 23 00:37:04.786745 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:04.786713    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64af7f56-51c6-4256-9957-cb2e84801629" path="/var/lib/kubelet/pods/64af7f56-51c6-4256-9957-cb2e84801629/volumes"
Jul 23 00:37:23.949117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:23.949085    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"]
Jul 23 00:37:23.949763 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:23.949419    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64af7f56-51c6-4256-9957-cb2e84801629" containerName="manager"
Jul 23 00:37:23.949763 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:23.949431    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="64af7f56-51c6-4256-9957-cb2e84801629" containerName="manager"
Jul 23 00:37:23.949763 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:23.949490    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="64af7f56-51c6-4256-9957-cb2e84801629" containerName="manager"
Jul 23 00:37:23.952151 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:23.952135    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:23.954779 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:23.954760    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 23 00:37:23.954892 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:23.954844    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sq2ww\""
Jul 23 00:37:23.963877 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:23.963854    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"]
Jul 23 00:37:24.030842 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.030812    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eee8f05c-892d-4bfd-8baf-604460a5a407-maas-api-tls\") pod \"maas-api-6d6b88c5c-mrx24\" (UID: \"eee8f05c-892d-4bfd-8baf-604460a5a407\") " pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:24.031032 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.030853    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eee8f05c-892d-4bfd-8baf-604460a5a407-openshift-service-ca\") pod \"maas-api-6d6b88c5c-mrx24\" (UID: \"eee8f05c-892d-4bfd-8baf-604460a5a407\") " pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:24.031032 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.030876    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v82x6\" (UniqueName: \"kubernetes.io/projected/eee8f05c-892d-4bfd-8baf-604460a5a407-kube-api-access-v82x6\") pod \"maas-api-6d6b88c5c-mrx24\" (UID: \"eee8f05c-892d-4bfd-8baf-604460a5a407\") " pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:24.131351 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.131314    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eee8f05c-892d-4bfd-8baf-604460a5a407-maas-api-tls\") pod \"maas-api-6d6b88c5c-mrx24\" (UID: \"eee8f05c-892d-4bfd-8baf-604460a5a407\") " pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:24.131351 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.131355    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eee8f05c-892d-4bfd-8baf-604460a5a407-openshift-service-ca\") pod \"maas-api-6d6b88c5c-mrx24\" (UID: \"eee8f05c-892d-4bfd-8baf-604460a5a407\") " pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:24.131659 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.131401    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v82x6\" (UniqueName: \"kubernetes.io/projected/eee8f05c-892d-4bfd-8baf-604460a5a407-kube-api-access-v82x6\") pod \"maas-api-6d6b88c5c-mrx24\" (UID: \"eee8f05c-892d-4bfd-8baf-604460a5a407\") " pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:24.132194 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.132051    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eee8f05c-892d-4bfd-8baf-604460a5a407-openshift-service-ca\") pod \"maas-api-6d6b88c5c-mrx24\" (UID: \"eee8f05c-892d-4bfd-8baf-604460a5a407\") " pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:24.137972 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.137942    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eee8f05c-892d-4bfd-8baf-604460a5a407-maas-api-tls\") pod \"maas-api-6d6b88c5c-mrx24\" (UID: \"eee8f05c-892d-4bfd-8baf-604460a5a407\") " pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:24.141639 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.141621    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v82x6\" (UniqueName: \"kubernetes.io/projected/eee8f05c-892d-4bfd-8baf-604460a5a407-kube-api-access-v82x6\") pod \"maas-api-6d6b88c5c-mrx24\" (UID: \"eee8f05c-892d-4bfd-8baf-604460a5a407\") " pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:24.262383 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.262290    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:24.405912 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.405870    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"]
Jul 23 00:37:24.408461 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:37:24.408433    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee8f05c_892d_4bfd_8baf_604460a5a407.slice/crio-c1c1ce17d109cf5d2d7c72d608ad92c931e8df37147542117c5658c300770a0d WatchSource:0}: Error finding container c1c1ce17d109cf5d2d7c72d608ad92c931e8df37147542117c5658c300770a0d: Status 404 returned error can't find the container with id c1c1ce17d109cf5d2d7c72d608ad92c931e8df37147542117c5658c300770a0d
Jul 23 00:37:24.513977 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.513899    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"]
Jul 23 00:37:24.540304 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.540273    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"]
Jul 23 00:37:24.540517 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.540421    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"
Jul 23 00:37:24.542910 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.542876    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lq9rt\""
Jul 23 00:37:24.542910 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.542892    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 23 00:37:24.555521 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.555500    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-2z2g7"]
Jul 23 00:37:24.571894 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.571871    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7"
Jul 23 00:37:24.572828 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.572804    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-2z2g7"]
Jul 23 00:37:24.636154 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.636116    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ded7055-6424-4dd9-8968-b6f3e7ecad22-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-zzxjj\" (UID: \"6ded7055-6424-4dd9-8968-b6f3e7ecad22\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"
Jul 23 00:37:24.636332 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.636213    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwhw8\" (UniqueName: \"kubernetes.io/projected/5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904-kube-api-access-zwhw8\") pod \"payload-processing-5bf47cd988-2z2g7\" (UID: \"5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904\") " pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7"
Jul 23 00:37:24.636332 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.636242    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904-plugins-config-volume\") pod \"payload-processing-5bf47cd988-2z2g7\" (UID: \"5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904\") " pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7"
Jul 23 00:37:24.636332 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.636276    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txw8l\" (UniqueName: \"kubernetes.io/projected/6ded7055-6424-4dd9-8968-b6f3e7ecad22-kube-api-access-txw8l\") pod \"payload-pre-processing-7677b774bb-zzxjj\" (UID: \"6ded7055-6424-4dd9-8968-b6f3e7ecad22\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"
Jul 23 00:37:24.737358 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.737313    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwhw8\" (UniqueName: \"kubernetes.io/projected/5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904-kube-api-access-zwhw8\") pod \"payload-processing-5bf47cd988-2z2g7\" (UID: \"5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904\") " pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7"
Jul 23 00:37:24.737559 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.737387    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904-plugins-config-volume\") pod \"payload-processing-5bf47cd988-2z2g7\" (UID: \"5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904\") " pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7"
Jul 23 00:37:24.737559 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.737423    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txw8l\" (UniqueName: \"kubernetes.io/projected/6ded7055-6424-4dd9-8968-b6f3e7ecad22-kube-api-access-txw8l\") pod \"payload-pre-processing-7677b774bb-zzxjj\" (UID: \"6ded7055-6424-4dd9-8968-b6f3e7ecad22\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"
Jul 23 00:37:24.737559 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.737466    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ded7055-6424-4dd9-8968-b6f3e7ecad22-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-zzxjj\" (UID: \"6ded7055-6424-4dd9-8968-b6f3e7ecad22\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"
Jul 23 00:37:24.738111 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.738090    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904-plugins-config-volume\") pod \"payload-processing-5bf47cd988-2z2g7\" (UID: \"5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904\") " pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7"
Jul 23 00:37:24.738200 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.738159    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ded7055-6424-4dd9-8968-b6f3e7ecad22-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-zzxjj\" (UID: \"6ded7055-6424-4dd9-8968-b6f3e7ecad22\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"
Jul 23 00:37:24.746012 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.745989    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwhw8\" (UniqueName: \"kubernetes.io/projected/5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904-kube-api-access-zwhw8\") pod \"payload-processing-5bf47cd988-2z2g7\" (UID: \"5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904\") " pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7"
Jul 23 00:37:24.746111 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.746057    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txw8l\" (UniqueName: \"kubernetes.io/projected/6ded7055-6424-4dd9-8968-b6f3e7ecad22-kube-api-access-txw8l\") pod \"payload-pre-processing-7677b774bb-zzxjj\" (UID: \"6ded7055-6424-4dd9-8968-b6f3e7ecad22\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"
Jul 23 00:37:24.851918 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.851836    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"
Jul 23 00:37:24.882044 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.882014    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7"
Jul 23 00:37:24.902893 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.902805    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24" event={"ID":"eee8f05c-892d-4bfd-8baf-604460a5a407","Type":"ContainerStarted","Data":"c1c1ce17d109cf5d2d7c72d608ad92c931e8df37147542117c5658c300770a0d"}
Jul 23 00:37:24.995101 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:24.995068    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"]
Jul 23 00:37:24.996902 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:37:24.996876    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ded7055_6424_4dd9_8968_b6f3e7ecad22.slice/crio-ac8c9e74b28f7523dea38741362798571fd65752527958c7708cbc6399132207 WatchSource:0}: Error finding container ac8c9e74b28f7523dea38741362798571fd65752527958c7708cbc6399132207: Status 404 returned error can't find the container with id ac8c9e74b28f7523dea38741362798571fd65752527958c7708cbc6399132207
Jul 23 00:37:25.029857 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:25.029709    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-2z2g7"]
Jul 23 00:37:25.032391 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:37:25.032343    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6ac6bb_d0f3_4cd7_8b7d_9ab2e2cd1904.slice/crio-7e8017255ae758a318b00c927788115a59b035f2418334177dbc4a41b3b5a843 WatchSource:0}: Error finding container 7e8017255ae758a318b00c927788115a59b035f2418334177dbc4a41b3b5a843: Status 404 returned error can't find the container with id 7e8017255ae758a318b00c927788115a59b035f2418334177dbc4a41b3b5a843
Jul 23 00:37:25.913075 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:25.912975    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7" event={"ID":"5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904","Type":"ContainerStarted","Data":"7e8017255ae758a318b00c927788115a59b035f2418334177dbc4a41b3b5a843"}
Jul 23 00:37:25.915144 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:25.915116    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj" event={"ID":"6ded7055-6424-4dd9-8968-b6f3e7ecad22","Type":"ContainerStarted","Data":"ac8c9e74b28f7523dea38741362798571fd65752527958c7708cbc6399132207"}
Jul 23 00:37:27.927854 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:27.927751    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj" event={"ID":"6ded7055-6424-4dd9-8968-b6f3e7ecad22","Type":"ContainerStarted","Data":"ac41658e3ebb0d0cff1935c745455a7ca4823caefcbc829396215224a34baeca"}
Jul 23 00:37:27.928290 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:27.927964    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"
Jul 23 00:37:27.929548 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:27.929516    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24" event={"ID":"eee8f05c-892d-4bfd-8baf-604460a5a407","Type":"ContainerStarted","Data":"8c469bee791dd1ae3e219a606c53ca3a9d9af53296c63ddb75577c462f6206e4"}
Jul 23 00:37:27.929693 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:27.929574    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:27.931097 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:27.931075    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7" event={"ID":"5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904","Type":"ContainerStarted","Data":"c0a947e0a52b276be08f79152b05e4f2ff9456e9c2f1c632d75d89490f4a5017"}
Jul 23 00:37:27.931232 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:27.931214    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7"
Jul 23 00:37:27.944249 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:27.944201    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj" podStartSLOduration=1.400773278 podStartE2EDuration="3.944188368s" totalImagesPullingTime="2.54341509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:37:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:37:27.942761143 +0000 UTC m=+851.749172377" watchObservedRunningTime="2026-07-23 00:37:27.944188368 +0000 UTC m=+851.750599601"
Jul 23 00:37:27.959798 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:27.959753    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24" podStartSLOduration=1.85855594 podStartE2EDuration="4.959739653s" totalImagesPullingTime="3.101183713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:37:27.959095401 +0000 UTC m=+851.765506638" watchObservedRunningTime="2026-07-23 00:37:27.959739653 +0000 UTC m=+851.766150903"
Jul 23 00:37:27.976116 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:27.976073    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7" podStartSLOduration=1.457140528 podStartE2EDuration="3.976060257s" totalImagesPullingTime="2.518919729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:37:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:37:27.975106442 +0000 UTC m=+851.781517666" watchObservedRunningTime="2026-07-23 00:37:27.976060257 +0000 UTC m=+851.782471489"
Jul 23 00:37:33.941258 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:33.941230    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d6b88c5c-mrx24"
Jul 23 00:37:38.936581 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:38.936550    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-2z2g7"
Jul 23 00:37:38.937043 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:37:38.936852    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-zzxjj"
Jul 23 00:38:00.089379 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.089339    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"]
Jul 23 00:38:00.099292 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.099261    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.102527 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.102502    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jtgsc\""
Jul 23 00:38:00.102527 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.102518    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 00:38:00.102734 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.102573    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 23 00:38:00.102734 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.102627    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 00:38:00.102837 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.102806    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"]
Jul 23 00:38:00.236196 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.236165    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a7740bf-117c-4293-9d1f-292fa3e40d94-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.236426 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.236223    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a7740bf-117c-4293-9d1f-292fa3e40d94-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.236426 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.236287    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a7740bf-117c-4293-9d1f-292fa3e40d94-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.236426 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.236312    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9cjs\" (UniqueName: \"kubernetes.io/projected/0a7740bf-117c-4293-9d1f-292fa3e40d94-kube-api-access-t9cjs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.236426 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.236341    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a7740bf-117c-4293-9d1f-292fa3e40d94-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.236426 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.236392    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a7740bf-117c-4293-9d1f-292fa3e40d94-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.336887 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.336848    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a7740bf-117c-4293-9d1f-292fa3e40d94-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.337065 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.336897    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a7740bf-117c-4293-9d1f-292fa3e40d94-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.337065 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.337035    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a7740bf-117c-4293-9d1f-292fa3e40d94-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.337175 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.337067    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9cjs\" (UniqueName: \"kubernetes.io/projected/0a7740bf-117c-4293-9d1f-292fa3e40d94-kube-api-access-t9cjs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.337175 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.337092    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a7740bf-117c-4293-9d1f-292fa3e40d94-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.337175 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.337118    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a7740bf-117c-4293-9d1f-292fa3e40d94-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.337323 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.337262    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a7740bf-117c-4293-9d1f-292fa3e40d94-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.337391 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.337315    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a7740bf-117c-4293-9d1f-292fa3e40d94-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.337391 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.337347    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a7740bf-117c-4293-9d1f-292fa3e40d94-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.339303 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.339273    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a7740bf-117c-4293-9d1f-292fa3e40d94-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.339689 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.339644    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a7740bf-117c-4293-9d1f-292fa3e40d94-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.345288 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.345265    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9cjs\" (UniqueName: \"kubernetes.io/projected/0a7740bf-117c-4293-9d1f-292fa3e40d94-kube-api-access-t9cjs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xx4m\" (UID: \"0a7740bf-117c-4293-9d1f-292fa3e40d94\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.410359 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.410328    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:00.544900 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:00.544877    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"]
Jul 23 00:38:00.547221 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:38:00.547194    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a7740bf_117c_4293_9d1f_292fa3e40d94.slice/crio-1afef1b3faf6162352766fe40122a45576e6860bbcdfd9a8d63909e28552e6b9 WatchSource:0}: Error finding container 1afef1b3faf6162352766fe40122a45576e6860bbcdfd9a8d63909e28552e6b9: Status 404 returned error can't find the container with id 1afef1b3faf6162352766fe40122a45576e6860bbcdfd9a8d63909e28552e6b9
Jul 23 00:38:01.085097 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:01.085014    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m" event={"ID":"0a7740bf-117c-4293-9d1f-292fa3e40d94","Type":"ContainerStarted","Data":"1afef1b3faf6162352766fe40122a45576e6860bbcdfd9a8d63909e28552e6b9"}
Jul 23 00:38:02.090334 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:02.090291    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m" event={"ID":"0a7740bf-117c-4293-9d1f-292fa3e40d94","Type":"ContainerStarted","Data":"fecc761200635d3e110683fdd95d1c9217cbf95cae667a2f85cc2e945b6f8050"}
Jul 23 00:38:02.108881 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:02.108821    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m" podStartSLOduration=0.977063251 podStartE2EDuration="2.108805289s" totalImagesPullingTime="1.131742038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:38:02.106279609 +0000 UTC m=+885.912690840" watchObservedRunningTime="2026-07-23 00:38:02.108805289 +0000 UTC m=+885.915216523"
Jul 23 00:38:10.411161 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:10.411073    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:10.411665 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:10.411162    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:10.423499 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:10.423477    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:11.141459 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:11.141430    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xx4m"
Jul 23 00:38:16.699829 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:16.699803    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:38:16.700241 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:38:16.699810    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:40:43.323720 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.323683    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-795c797fc6-54kwp"]
Jul 23 00:40:43.324164 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.323943    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-795c797fc6-54kwp" podUID="5d39aa6e-fbd2-48c8-b7ba-8b614abce076" containerName="manager" containerID="cri-o://946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c" gracePeriod=30
Jul 23 00:40:43.573066 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.573041    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:40:43.677551 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.677454    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rj8m9\" (UniqueName: \"kubernetes.io/projected/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-kube-api-access-rj8m9\") pod \"5d39aa6e-fbd2-48c8-b7ba-8b614abce076\" (UID: \"5d39aa6e-fbd2-48c8-b7ba-8b614abce076\") "
Jul 23 00:40:43.677725 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.677550    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-webhook-cert\") pod \"5d39aa6e-fbd2-48c8-b7ba-8b614abce076\" (UID: \"5d39aa6e-fbd2-48c8-b7ba-8b614abce076\") "
Jul 23 00:40:43.679540 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.679507    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-kube-api-access-rj8m9" (OuterVolumeSpecName: "kube-api-access-rj8m9") pod "5d39aa6e-fbd2-48c8-b7ba-8b614abce076" (UID: "5d39aa6e-fbd2-48c8-b7ba-8b614abce076"). InnerVolumeSpecName "kube-api-access-rj8m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:40:43.679641 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.679513    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5d39aa6e-fbd2-48c8-b7ba-8b614abce076" (UID: "5d39aa6e-fbd2-48c8-b7ba-8b614abce076"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:40:43.778589 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.778557    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rj8m9\" (UniqueName: \"kubernetes.io/projected/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-kube-api-access-rj8m9\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:40:43.778589 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.778583    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5d39aa6e-fbd2-48c8-b7ba-8b614abce076-webhook-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:40:43.819921 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.819892    2561 generic.go:356] "Generic (PLEG): container finished" podID="5d39aa6e-fbd2-48c8-b7ba-8b614abce076" containerID="946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c" exitCode=0
Jul 23 00:40:43.820108 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.819950    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-795c797fc6-54kwp"
Jul 23 00:40:43.820108 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.819985    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795c797fc6-54kwp" event={"ID":"5d39aa6e-fbd2-48c8-b7ba-8b614abce076","Type":"ContainerDied","Data":"946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c"}
Jul 23 00:40:43.820108 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.820039    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795c797fc6-54kwp" event={"ID":"5d39aa6e-fbd2-48c8-b7ba-8b614abce076","Type":"ContainerDied","Data":"32b5cb9fff6ec1f2171299b978b3cbfbe8a8135ce4cf9a02ffd80eec6c86e8f6"}
Jul 23 00:40:43.820108 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.820062    2561 scope.go:117] "RemoveContainer" containerID="946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c"
Jul 23 00:40:43.830674 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.830648    2561 scope.go:117] "RemoveContainer" containerID="946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c"
Jul 23 00:40:43.831145 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:40:43.831121    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c\": container with ID starting with 946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c not found: ID does not exist" containerID="946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c"
Jul 23 00:40:43.831235 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.831153    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c"} err="failed to get container status \"946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c\": rpc error: code = NotFound desc = could not find container \"946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c\": container with ID starting with 946755d77f5c07428c303f27347db73eadb3898e3f17e5b5c86b6b1ad0450d6c not found: ID does not exist"
Jul 23 00:40:43.844134 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.844108    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-795c797fc6-54kwp"]
Jul 23 00:40:43.851110 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:43.851088    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-795c797fc6-54kwp"]
Jul 23 00:40:44.786968 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:40:44.786921    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d39aa6e-fbd2-48c8-b7ba-8b614abce076" path="/var/lib/kubelet/pods/5d39aa6e-fbd2-48c8-b7ba-8b614abce076/volumes"
Jul 23 00:41:44.642793 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.642758    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-795c797fc6-x97v8"]
Jul 23 00:41:44.643212 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.643117    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d39aa6e-fbd2-48c8-b7ba-8b614abce076" containerName="manager"
Jul 23 00:41:44.643212 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.643129    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d39aa6e-fbd2-48c8-b7ba-8b614abce076" containerName="manager"
Jul 23 00:41:44.643212 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.643192    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d39aa6e-fbd2-48c8-b7ba-8b614abce076" containerName="manager"
Jul 23 00:41:44.645893 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.645877    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-795c797fc6-x97v8"
Jul 23 00:41:44.652943 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.652916    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 00:41:44.652943 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.652929    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kzpcs\""
Jul 23 00:41:44.653152 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.652959    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 00:41:44.655734 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.655713    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-795c797fc6-x97v8"]
Jul 23 00:41:44.686645 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.686614    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37b3fac9-ac22-473b-adf7-a8a862ae5f3f-webhook-cert\") pod \"maas-controller-795c797fc6-x97v8\" (UID: \"37b3fac9-ac22-473b-adf7-a8a862ae5f3f\") " pod="opendatahub/maas-controller-795c797fc6-x97v8"
Jul 23 00:41:44.686791 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.686649    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5zkh\" (UniqueName: \"kubernetes.io/projected/37b3fac9-ac22-473b-adf7-a8a862ae5f3f-kube-api-access-p5zkh\") pod \"maas-controller-795c797fc6-x97v8\" (UID: \"37b3fac9-ac22-473b-adf7-a8a862ae5f3f\") " pod="opendatahub/maas-controller-795c797fc6-x97v8"
Jul 23 00:41:44.787537 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.787504    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37b3fac9-ac22-473b-adf7-a8a862ae5f3f-webhook-cert\") pod \"maas-controller-795c797fc6-x97v8\" (UID: \"37b3fac9-ac22-473b-adf7-a8a862ae5f3f\") " pod="opendatahub/maas-controller-795c797fc6-x97v8"
Jul 23 00:41:44.787537 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.787533    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5zkh\" (UniqueName: \"kubernetes.io/projected/37b3fac9-ac22-473b-adf7-a8a862ae5f3f-kube-api-access-p5zkh\") pod \"maas-controller-795c797fc6-x97v8\" (UID: \"37b3fac9-ac22-473b-adf7-a8a862ae5f3f\") " pod="opendatahub/maas-controller-795c797fc6-x97v8"
Jul 23 00:41:44.789918 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.789895    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37b3fac9-ac22-473b-adf7-a8a862ae5f3f-webhook-cert\") pod \"maas-controller-795c797fc6-x97v8\" (UID: \"37b3fac9-ac22-473b-adf7-a8a862ae5f3f\") " pod="opendatahub/maas-controller-795c797fc6-x97v8"
Jul 23 00:41:44.795430 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.795407    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5zkh\" (UniqueName: \"kubernetes.io/projected/37b3fac9-ac22-473b-adf7-a8a862ae5f3f-kube-api-access-p5zkh\") pod \"maas-controller-795c797fc6-x97v8\" (UID: \"37b3fac9-ac22-473b-adf7-a8a862ae5f3f\") " pod="opendatahub/maas-controller-795c797fc6-x97v8"
Jul 23 00:41:44.963861 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:44.963833    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-795c797fc6-x97v8"
Jul 23 00:41:45.093748 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:45.093725    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-795c797fc6-x97v8"]
Jul 23 00:41:45.095523 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:41:45.095497    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37b3fac9_ac22_473b_adf7_a8a862ae5f3f.slice/crio-4bed1785972340b054fc707365d03a6a4713eaeb67b10b72268d7188b0067365 WatchSource:0}: Error finding container 4bed1785972340b054fc707365d03a6a4713eaeb67b10b72268d7188b0067365: Status 404 returned error can't find the container with id 4bed1785972340b054fc707365d03a6a4713eaeb67b10b72268d7188b0067365
Jul 23 00:41:45.096831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:45.096815    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 00:41:46.092100 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:46.092068    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795c797fc6-x97v8" event={"ID":"37b3fac9-ac22-473b-adf7-a8a862ae5f3f","Type":"ContainerStarted","Data":"4bed1785972340b054fc707365d03a6a4713eaeb67b10b72268d7188b0067365"}
Jul 23 00:41:48.102896 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:48.102807    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795c797fc6-x97v8" event={"ID":"37b3fac9-ac22-473b-adf7-a8a862ae5f3f","Type":"ContainerStarted","Data":"24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293"}
Jul 23 00:41:48.102896 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:48.102869    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-795c797fc6-x97v8"
Jul 23 00:41:48.120214 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:48.120166    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-795c797fc6-x97v8" podStartSLOduration=1.4170231389999999 podStartE2EDuration="4.120151191s" totalImagesPullingTime="2.703128052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:41:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:41:48.118599866 +0000 UTC m=+1111.925011111" watchObservedRunningTime="2026-07-23 00:41:48.120151191 +0000 UTC m=+1111.926562423"
Jul 23 00:41:59.109151 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:41:59.109120    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-795c797fc6-x97v8"
Jul 23 00:42:03.768405 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:03.768355    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-795c797fc6-x97v8"]
Jul 23 00:42:03.768853 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:03.768561    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-795c797fc6-x97v8" podUID="37b3fac9-ac22-473b-adf7-a8a862ae5f3f" containerName="manager" containerID="cri-o://24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293" gracePeriod=30
Jul 23 00:42:04.017486 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.017463    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-795c797fc6-x97v8"
Jul 23 00:42:04.159531 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.159429    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37b3fac9-ac22-473b-adf7-a8a862ae5f3f-webhook-cert\") pod \"37b3fac9-ac22-473b-adf7-a8a862ae5f3f\" (UID: \"37b3fac9-ac22-473b-adf7-a8a862ae5f3f\") "
Jul 23 00:42:04.159531 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.159486    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5zkh\" (UniqueName: \"kubernetes.io/projected/37b3fac9-ac22-473b-adf7-a8a862ae5f3f-kube-api-access-p5zkh\") pod \"37b3fac9-ac22-473b-adf7-a8a862ae5f3f\" (UID: \"37b3fac9-ac22-473b-adf7-a8a862ae5f3f\") "
Jul 23 00:42:04.161626 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.161596    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37b3fac9-ac22-473b-adf7-a8a862ae5f3f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "37b3fac9-ac22-473b-adf7-a8a862ae5f3f" (UID: "37b3fac9-ac22-473b-adf7-a8a862ae5f3f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:42:04.161626 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.161616    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37b3fac9-ac22-473b-adf7-a8a862ae5f3f-kube-api-access-p5zkh" (OuterVolumeSpecName: "kube-api-access-p5zkh") pod "37b3fac9-ac22-473b-adf7-a8a862ae5f3f" (UID: "37b3fac9-ac22-473b-adf7-a8a862ae5f3f"). InnerVolumeSpecName "kube-api-access-p5zkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:42:04.176134 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.176102    2561 generic.go:356] "Generic (PLEG): container finished" podID="37b3fac9-ac22-473b-adf7-a8a862ae5f3f" containerID="24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293" exitCode=0
Jul 23 00:42:04.176288 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.176140    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795c797fc6-x97v8" event={"ID":"37b3fac9-ac22-473b-adf7-a8a862ae5f3f","Type":"ContainerDied","Data":"24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293"}
Jul 23 00:42:04.176288 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.176163    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-795c797fc6-x97v8"
Jul 23 00:42:04.176288 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.176217    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795c797fc6-x97v8" event={"ID":"37b3fac9-ac22-473b-adf7-a8a862ae5f3f","Type":"ContainerDied","Data":"4bed1785972340b054fc707365d03a6a4713eaeb67b10b72268d7188b0067365"}
Jul 23 00:42:04.176288 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.176251    2561 scope.go:117] "RemoveContainer" containerID="24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293"
Jul 23 00:42:04.186066 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.186049    2561 scope.go:117] "RemoveContainer" containerID="24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293"
Jul 23 00:42:04.186320 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:42:04.186301    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293\": container with ID starting with 24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293 not found: ID does not exist" containerID="24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293"
Jul 23 00:42:04.186435 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.186333    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293"} err="failed to get container status \"24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293\": rpc error: code = NotFound desc = could not find container \"24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293\": container with ID starting with 24fe172ce4a9ccbd8f39dc666440250ba22a07b4507570b8294fb9079bd4f293 not found: ID does not exist"
Jul 23 00:42:04.197558 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.197532    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-795c797fc6-x97v8"]
Jul 23 00:42:04.201405 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.201385    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-795c797fc6-x97v8"]
Jul 23 00:42:04.261078 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.261043    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37b3fac9-ac22-473b-adf7-a8a862ae5f3f-webhook-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:42:04.261078 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.261071    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5zkh\" (UniqueName: \"kubernetes.io/projected/37b3fac9-ac22-473b-adf7-a8a862ae5f3f-kube-api-access-p5zkh\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:42:04.787331 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:04.787298    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37b3fac9-ac22-473b-adf7-a8a862ae5f3f" path="/var/lib/kubelet/pods/37b3fac9-ac22-473b-adf7-a8a862ae5f3f/volumes"
Jul 23 00:42:05.005661 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.005632    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-795c797fc6-c5v62"]
Jul 23 00:42:05.005957 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.005944    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37b3fac9-ac22-473b-adf7-a8a862ae5f3f" containerName="manager"
Jul 23 00:42:05.006016 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.005958    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="37b3fac9-ac22-473b-adf7-a8a862ae5f3f" containerName="manager"
Jul 23 00:42:05.006058 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.006028    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="37b3fac9-ac22-473b-adf7-a8a862ae5f3f" containerName="manager"
Jul 23 00:42:05.010172 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.010152    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-795c797fc6-c5v62"
Jul 23 00:42:05.012652 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.012633    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 00:42:05.013634 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.013617    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kzpcs\""
Jul 23 00:42:05.013708 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.013673    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 00:42:05.018990 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.018959    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-795c797fc6-c5v62"]
Jul 23 00:42:05.168254 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.168171    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b94b6c9a-67da-4989-a5d3-131b1b3352e3-webhook-cert\") pod \"maas-controller-795c797fc6-c5v62\" (UID: \"b94b6c9a-67da-4989-a5d3-131b1b3352e3\") " pod="opendatahub/maas-controller-795c797fc6-c5v62"
Jul 23 00:42:05.168421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.168260    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m278l\" (UniqueName: \"kubernetes.io/projected/b94b6c9a-67da-4989-a5d3-131b1b3352e3-kube-api-access-m278l\") pod \"maas-controller-795c797fc6-c5v62\" (UID: \"b94b6c9a-67da-4989-a5d3-131b1b3352e3\") " pod="opendatahub/maas-controller-795c797fc6-c5v62"
Jul 23 00:42:05.269603 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.269568    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m278l\" (UniqueName: \"kubernetes.io/projected/b94b6c9a-67da-4989-a5d3-131b1b3352e3-kube-api-access-m278l\") pod \"maas-controller-795c797fc6-c5v62\" (UID: \"b94b6c9a-67da-4989-a5d3-131b1b3352e3\") " pod="opendatahub/maas-controller-795c797fc6-c5v62"
Jul 23 00:42:05.269802 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.269617    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b94b6c9a-67da-4989-a5d3-131b1b3352e3-webhook-cert\") pod \"maas-controller-795c797fc6-c5v62\" (UID: \"b94b6c9a-67da-4989-a5d3-131b1b3352e3\") " pod="opendatahub/maas-controller-795c797fc6-c5v62"
Jul 23 00:42:05.271968 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.271945    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b94b6c9a-67da-4989-a5d3-131b1b3352e3-webhook-cert\") pod \"maas-controller-795c797fc6-c5v62\" (UID: \"b94b6c9a-67da-4989-a5d3-131b1b3352e3\") " pod="opendatahub/maas-controller-795c797fc6-c5v62"
Jul 23 00:42:05.277242 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.277223    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m278l\" (UniqueName: \"kubernetes.io/projected/b94b6c9a-67da-4989-a5d3-131b1b3352e3-kube-api-access-m278l\") pod \"maas-controller-795c797fc6-c5v62\" (UID: \"b94b6c9a-67da-4989-a5d3-131b1b3352e3\") " pod="opendatahub/maas-controller-795c797fc6-c5v62"
Jul 23 00:42:05.321262 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.321223    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-795c797fc6-c5v62"
Jul 23 00:42:05.457145 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:05.456206    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-795c797fc6-c5v62"]
Jul 23 00:42:06.188316 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:06.188273    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795c797fc6-c5v62" event={"ID":"b94b6c9a-67da-4989-a5d3-131b1b3352e3","Type":"ContainerStarted","Data":"afcbf675c6f66a44474ca60ba7698219ec3cbdc35922a9a9c9b378f69d3e9413"}
Jul 23 00:42:06.188819 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:06.188333    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795c797fc6-c5v62" event={"ID":"b94b6c9a-67da-4989-a5d3-131b1b3352e3","Type":"ContainerStarted","Data":"ef9ef5aa49e408a63b17615ff398c3f80f1ad12e51f24224be3929939d06429b"}
Jul 23 00:42:06.188819 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:06.188428    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-795c797fc6-c5v62"
Jul 23 00:42:06.206261 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:06.206205    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-795c797fc6-c5v62" podStartSLOduration=1.6795144450000001 podStartE2EDuration="2.206190018s" totalImagesPullingTime="526.675573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:42:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:42:06.204334166 +0000 UTC m=+1130.010745399" watchObservedRunningTime="2026-07-23 00:42:06.206190018 +0000 UTC m=+1130.012601251"
Jul 23 00:42:17.194873 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:42:17.194841    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-795c797fc6-c5v62"
Jul 23 00:43:16.725541 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:43:16.725508    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:43:16.726728 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:43:16.726700    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:45:00.146933 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:00.146893    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746125-2mtj8"]
Jul 23 00:45:00.150018 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:00.150002    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746125-2mtj8"
Jul 23 00:45:00.152756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:00.152736    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4zd6c\""
Jul 23 00:45:00.159196 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:00.159180    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746125-2mtj8"
Jul 23 00:45:00.169171 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:00.169144    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746125-2mtj8"]
Jul 23 00:45:00.289591 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:00.289562    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746125-2mtj8"]
Jul 23 00:45:00.291444 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:45:00.291418    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a66aea_b659_4149_9457_7f18e46602a8.slice/crio-3ce824f3a3c5ee72fb9a34941e89e2b48a268642822d69ea1c45807a39160bb3 WatchSource:0}: Error finding container 3ce824f3a3c5ee72fb9a34941e89e2b48a268642822d69ea1c45807a39160bb3: Status 404 returned error can't find the container with id 3ce824f3a3c5ee72fb9a34941e89e2b48a268642822d69ea1c45807a39160bb3
Jul 23 00:45:00.964770 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:00.964736    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746125-2mtj8" event={"ID":"32a66aea-b659-4149-9457-7f18e46602a8","Type":"ContainerStarted","Data":"3ce824f3a3c5ee72fb9a34941e89e2b48a268642822d69ea1c45807a39160bb3"}
Jul 23 00:45:02.974784 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:02.974754    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746125-2mtj8" event={"ID":"32a66aea-b659-4149-9457-7f18e46602a8","Type":"ContainerStarted","Data":"25069312f1f6e6cf752657930e970a1b17b73fa86139641cf9e0536115f6e8ac"}
Jul 23 00:45:02.990837 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:02.990789    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746125-2mtj8" podStartSLOduration=0.808679123 podStartE2EDuration="2.990775601s" totalImagesPullingTime="2.182096478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:45:02.989274473 +0000 UTC m=+1306.795685709" watchObservedRunningTime="2026-07-23 00:45:02.990775601 +0000 UTC m=+1306.797186836"
Jul 23 00:45:03.980232 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:03.980202    2561 generic.go:356] "Generic (PLEG): container finished" podID="32a66aea-b659-4149-9457-7f18e46602a8" containerID="25069312f1f6e6cf752657930e970a1b17b73fa86139641cf9e0536115f6e8ac" exitCode=0
Jul 23 00:45:03.980635 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:03.980276    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746125-2mtj8" event={"ID":"32a66aea-b659-4149-9457-7f18e46602a8","Type":"ContainerDied","Data":"25069312f1f6e6cf752657930e970a1b17b73fa86139641cf9e0536115f6e8ac"}
Jul 23 00:45:05.121036 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:05.121013    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746125-2mtj8"
Jul 23 00:45:05.990950 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:05.990917    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746125-2mtj8"
Jul 23 00:45:05.991129 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:05.990956    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746125-2mtj8" event={"ID":"32a66aea-b659-4149-9457-7f18e46602a8","Type":"ContainerDied","Data":"3ce824f3a3c5ee72fb9a34941e89e2b48a268642822d69ea1c45807a39160bb3"}
Jul 23 00:45:05.991129 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:45:05.991004    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ce824f3a3c5ee72fb9a34941e89e2b48a268642822d69ea1c45807a39160bb3"
Jul 23 00:46:51.526766 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:51.526726    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"]
Jul 23 00:46:51.527447 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:51.526967    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx" podUID="9033bade-2184-4fd4-84ce-db1f703471d9" containerName="manager" containerID="cri-o://ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5" gracePeriod=10
Jul 23 00:46:54.080709 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.080679    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"
Jul 23 00:46:54.178425 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.178312    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9033bade-2184-4fd4-84ce-db1f703471d9-extensions-socket-volume\") pod \"9033bade-2184-4fd4-84ce-db1f703471d9\" (UID: \"9033bade-2184-4fd4-84ce-db1f703471d9\") "
Jul 23 00:46:54.178596 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.178429    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nvnc\" (UniqueName: \"kubernetes.io/projected/9033bade-2184-4fd4-84ce-db1f703471d9-kube-api-access-2nvnc\") pod \"9033bade-2184-4fd4-84ce-db1f703471d9\" (UID: \"9033bade-2184-4fd4-84ce-db1f703471d9\") "
Jul 23 00:46:54.178774 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.178751    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9033bade-2184-4fd4-84ce-db1f703471d9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9033bade-2184-4fd4-84ce-db1f703471d9" (UID: "9033bade-2184-4fd4-84ce-db1f703471d9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:46:54.180421 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.180390    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9033bade-2184-4fd4-84ce-db1f703471d9-kube-api-access-2nvnc" (OuterVolumeSpecName: "kube-api-access-2nvnc") pod "9033bade-2184-4fd4-84ce-db1f703471d9" (UID: "9033bade-2184-4fd4-84ce-db1f703471d9"). InnerVolumeSpecName "kube-api-access-2nvnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:46:54.279700 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.279665    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9033bade-2184-4fd4-84ce-db1f703471d9-extensions-socket-volume\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:46:54.279700 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.279692    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nvnc\" (UniqueName: \"kubernetes.io/projected/9033bade-2184-4fd4-84ce-db1f703471d9-kube-api-access-2nvnc\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:46:54.476240 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.476205    2561 generic.go:356] "Generic (PLEG): container finished" podID="9033bade-2184-4fd4-84ce-db1f703471d9" containerID="ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5" exitCode=0
Jul 23 00:46:54.476451 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.476252    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"
Jul 23 00:46:54.476451 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.476297    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx" event={"ID":"9033bade-2184-4fd4-84ce-db1f703471d9","Type":"ContainerDied","Data":"ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5"}
Jul 23 00:46:54.476451 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.476407    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx" event={"ID":"9033bade-2184-4fd4-84ce-db1f703471d9","Type":"ContainerDied","Data":"3d45af9db75ded04dcb71e8666cb5cd402f61949cdefe79b8c0858b9d8b6f582"}
Jul 23 00:46:54.476451 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.476431    2561 scope.go:117] "RemoveContainer" containerID="ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5"
Jul 23 00:46:54.485536 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.485520    2561 scope.go:117] "RemoveContainer" containerID="ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5"
Jul 23 00:46:54.485791 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:46:54.485770    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5\": container with ID starting with ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5 not found: ID does not exist" containerID="ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5"
Jul 23 00:46:54.485831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.485801    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5"} err="failed to get container status \"ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5\": rpc error: code = NotFound desc = could not find container \"ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5\": container with ID starting with ac8467d55e59f31a602aec6a3cf601bf028a190a5fd749a73b6b67838ecd23c5 not found: ID does not exist"
Jul 23 00:46:54.498192 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.498168    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"]
Jul 23 00:46:54.502003 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.501982    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l6fjx"]
Jul 23 00:46:54.788964 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:46:54.788885    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9033bade-2184-4fd4-84ce-db1f703471d9" path="/var/lib/kubelet/pods/9033bade-2184-4fd4-84ce-db1f703471d9/volumes"
Jul 23 00:47:52.704496 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.704459    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"]
Jul 23 00:47:52.704955 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.704797    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32a66aea-b659-4149-9457-7f18e46602a8" containerName="cleanup"
Jul 23 00:47:52.704955 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.704808    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="32a66aea-b659-4149-9457-7f18e46602a8" containerName="cleanup"
Jul 23 00:47:52.704955 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.704822    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9033bade-2184-4fd4-84ce-db1f703471d9" containerName="manager"
Jul 23 00:47:52.704955 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.704827    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9033bade-2184-4fd4-84ce-db1f703471d9" containerName="manager"
Jul 23 00:47:52.704955 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.704882    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="32a66aea-b659-4149-9457-7f18e46602a8" containerName="cleanup"
Jul 23 00:47:52.704955 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.704892    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9033bade-2184-4fd4-84ce-db1f703471d9" containerName="manager"
Jul 23 00:47:52.707858 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.707842    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"
Jul 23 00:47:52.710569 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.710548    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hdqfk\""
Jul 23 00:47:52.728286 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.728264    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"]
Jul 23 00:47:52.866299 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.866264    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk65d\" (UniqueName: \"kubernetes.io/projected/6e7e0f2c-3558-4054-b460-2da581eb40fd-kube-api-access-dk65d\") pod \"kuadrant-operator-controller-manager-86878c769f-bfx9c\" (UID: \"6e7e0f2c-3558-4054-b460-2da581eb40fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"
Jul 23 00:47:52.866500 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.866323    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e7e0f2c-3558-4054-b460-2da581eb40fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bfx9c\" (UID: \"6e7e0f2c-3558-4054-b460-2da581eb40fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"
Jul 23 00:47:52.967325 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.967231    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e7e0f2c-3558-4054-b460-2da581eb40fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bfx9c\" (UID: \"6e7e0f2c-3558-4054-b460-2da581eb40fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"
Jul 23 00:47:52.967325 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.967323    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk65d\" (UniqueName: \"kubernetes.io/projected/6e7e0f2c-3558-4054-b460-2da581eb40fd-kube-api-access-dk65d\") pod \"kuadrant-operator-controller-manager-86878c769f-bfx9c\" (UID: \"6e7e0f2c-3558-4054-b460-2da581eb40fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"
Jul 23 00:47:52.967627 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.967606    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e7e0f2c-3558-4054-b460-2da581eb40fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bfx9c\" (UID: \"6e7e0f2c-3558-4054-b460-2da581eb40fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"
Jul 23 00:47:52.977768 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:52.977741    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk65d\" (UniqueName: \"kubernetes.io/projected/6e7e0f2c-3558-4054-b460-2da581eb40fd-kube-api-access-dk65d\") pod \"kuadrant-operator-controller-manager-86878c769f-bfx9c\" (UID: \"6e7e0f2c-3558-4054-b460-2da581eb40fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"
Jul 23 00:47:53.017781 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:53.017747    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"
Jul 23 00:47:53.151897 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:53.151874    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"]
Jul 23 00:47:53.154462 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:47:53.154429    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e7e0f2c_3558_4054_b460_2da581eb40fd.slice/crio-4c4bd638d093c788633b7752070417dad75b16fb22d22c736b6cb5515aa66063 WatchSource:0}: Error finding container 4c4bd638d093c788633b7752070417dad75b16fb22d22c736b6cb5515aa66063: Status 404 returned error can't find the container with id 4c4bd638d093c788633b7752070417dad75b16fb22d22c736b6cb5515aa66063
Jul 23 00:47:53.733475 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:53.733439    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c" event={"ID":"6e7e0f2c-3558-4054-b460-2da581eb40fd","Type":"ContainerStarted","Data":"17e132a9ee864e160ca23fc814ccff2fac452055800c7c46efdaa6a5058b8b59"}
Jul 23 00:47:53.733880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:53.733489    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c" event={"ID":"6e7e0f2c-3558-4054-b460-2da581eb40fd","Type":"ContainerStarted","Data":"4c4bd638d093c788633b7752070417dad75b16fb22d22c736b6cb5515aa66063"}
Jul 23 00:47:53.733880 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:53.733519    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"
Jul 23 00:47:53.754321 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:47:53.754269    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c" podStartSLOduration=1.7542547590000002 podStartE2EDuration="1.754254759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:47:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:47:53.752124154 +0000 UTC m=+1477.558535387" watchObservedRunningTime="2026-07-23 00:47:53.754254759 +0000 UTC m=+1477.560666051"
Jul 23 00:48:04.740393 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:48:04.740338    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bfx9c"
Jul 23 00:48:16.753141 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:48:16.753107    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:48:16.753851 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:48:16.753835    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:50:32.902295 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:32.902220    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"]
Jul 23 00:50:32.905380 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:32.905345    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:32.908331 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:32.908306    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-c49d048f-kserve-self-signed-certs\""
Jul 23 00:50:32.925171 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:32.925149    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"]
Jul 23 00:50:33.006293 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.006257    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-home\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.006293 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.006296    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-dshm\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.006523 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.006329    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl7gd\" (UniqueName: \"kubernetes.io/projected/fe307395-2fd4-4977-afeb-1105339adedc-kube-api-access-xl7gd\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.006523 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.006413    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-model-cache\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.006523 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.006457    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-tmp-dir\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.006523 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.006520    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe307395-2fd4-4977-afeb-1105339adedc-tls-certs\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.107144 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.107107    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-home\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.107144 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.107143    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-dshm\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.107439 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.107163    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl7gd\" (UniqueName: \"kubernetes.io/projected/fe307395-2fd4-4977-afeb-1105339adedc-kube-api-access-xl7gd\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.107439 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.107184    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-model-cache\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.107439 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.107212    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-tmp-dir\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.107439 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.107338    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe307395-2fd4-4977-afeb-1105339adedc-tls-certs\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.107668 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.107602    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-home\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.107668 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.107619    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-model-cache\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.107741 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.107704    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-tmp-dir\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.109528 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.109508    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-dshm\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.109689 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.109673    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe307395-2fd4-4977-afeb-1105339adedc-tls-certs\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.123992 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.123961    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl7gd\" (UniqueName: \"kubernetes.io/projected/fe307395-2fd4-4977-afeb-1105339adedc-kube-api-access-xl7gd\") pod \"e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") " pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.214743 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.214711    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:33.350182 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.350064    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"]
Jul 23 00:50:33.352440 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:50:33.352414    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe307395_2fd4_4977_afeb_1105339adedc.slice/crio-b693307ac561fae42ae19bf5c9e591076b69e5ec05933324eecc9f82149c1c46 WatchSource:0}: Error finding container b693307ac561fae42ae19bf5c9e591076b69e5ec05933324eecc9f82149c1c46: Status 404 returned error can't find the container with id b693307ac561fae42ae19bf5c9e591076b69e5ec05933324eecc9f82149c1c46
Jul 23 00:50:33.454443 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.454408    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2" event={"ID":"fe307395-2fd4-4977-afeb-1105339adedc","Type":"ContainerStarted","Data":"c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76"}
Jul 23 00:50:33.454443 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.454448    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2" event={"ID":"fe307395-2fd4-4977-afeb-1105339adedc","Type":"ContainerStarted","Data":"b693307ac561fae42ae19bf5c9e591076b69e5ec05933324eecc9f82149c1c46"}
Jul 23 00:50:33.476616 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:33.476499    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2" podStartSLOduration=1.476478162 podStartE2EDuration="1.476478162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:50:33.473628846 +0000 UTC m=+1637.280040080" watchObservedRunningTime="2026-07-23 00:50:33.476478162 +0000 UTC m=+1637.282889395"
Jul 23 00:50:43.215850 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:43.215810    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:43.216422 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:43.215862    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:43.229277 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:43.229252    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:50:43.510083 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:50:43.510001    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:51:18.158355 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:18.158318    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"]
Jul 23 00:51:33.179829 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.179759    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2" podUID="fe307395-2fd4-4977-afeb-1105339adedc" containerName="main" containerID="cri-o://c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76" gracePeriod=45
Jul 23 00:51:33.434268 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.434245    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:51:33.502542 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.502514    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe307395-2fd4-4977-afeb-1105339adedc-tls-certs\") pod \"fe307395-2fd4-4977-afeb-1105339adedc\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") "
Jul 23 00:51:33.502714 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.502555    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-home\") pod \"fe307395-2fd4-4977-afeb-1105339adedc\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") "
Jul 23 00:51:33.502714 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.502600    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-tmp-dir\") pod \"fe307395-2fd4-4977-afeb-1105339adedc\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") "
Jul 23 00:51:33.502714 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.502619    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-dshm\") pod \"fe307395-2fd4-4977-afeb-1105339adedc\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") "
Jul 23 00:51:33.502714 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.502637    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-model-cache\") pod \"fe307395-2fd4-4977-afeb-1105339adedc\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") "
Jul 23 00:51:33.502714 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.502677    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xl7gd\" (UniqueName: \"kubernetes.io/projected/fe307395-2fd4-4977-afeb-1105339adedc-kube-api-access-xl7gd\") pod \"fe307395-2fd4-4977-afeb-1105339adedc\" (UID: \"fe307395-2fd4-4977-afeb-1105339adedc\") "
Jul 23 00:51:33.502989 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.502892    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fe307395-2fd4-4977-afeb-1105339adedc" (UID: "fe307395-2fd4-4977-afeb-1105339adedc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:51:33.502989 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.502950    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-home" (OuterVolumeSpecName: "home") pod "fe307395-2fd4-4977-afeb-1105339adedc" (UID: "fe307395-2fd4-4977-afeb-1105339adedc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:51:33.502989 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.502966    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-model-cache" (OuterVolumeSpecName: "model-cache") pod "fe307395-2fd4-4977-afeb-1105339adedc" (UID: "fe307395-2fd4-4977-afeb-1105339adedc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:51:33.504865 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.504836    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-dshm" (OuterVolumeSpecName: "dshm") pod "fe307395-2fd4-4977-afeb-1105339adedc" (UID: "fe307395-2fd4-4977-afeb-1105339adedc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:51:33.504981 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.504890    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe307395-2fd4-4977-afeb-1105339adedc-kube-api-access-xl7gd" (OuterVolumeSpecName: "kube-api-access-xl7gd") pod "fe307395-2fd4-4977-afeb-1105339adedc" (UID: "fe307395-2fd4-4977-afeb-1105339adedc"). InnerVolumeSpecName "kube-api-access-xl7gd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:51:33.504981 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.504907    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe307395-2fd4-4977-afeb-1105339adedc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fe307395-2fd4-4977-afeb-1105339adedc" (UID: "fe307395-2fd4-4977-afeb-1105339adedc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:51:33.603358 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.603327    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-home\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:51:33.603358 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.603355    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-tmp-dir\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:51:33.603568 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.603393    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-dshm\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:51:33.603568 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.603403    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe307395-2fd4-4977-afeb-1105339adedc-model-cache\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:51:33.603568 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.603412    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xl7gd\" (UniqueName: \"kubernetes.io/projected/fe307395-2fd4-4977-afeb-1105339adedc-kube-api-access-xl7gd\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:51:33.603568 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.603420    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe307395-2fd4-4977-afeb-1105339adedc-tls-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:51:33.722962 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.722870    2561 generic.go:356] "Generic (PLEG): container finished" podID="fe307395-2fd4-4977-afeb-1105339adedc" containerID="c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76" exitCode=0
Jul 23 00:51:33.722962 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.722916    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"
Jul 23 00:51:33.723201 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.722964    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2" event={"ID":"fe307395-2fd4-4977-afeb-1105339adedc","Type":"ContainerDied","Data":"c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76"}
Jul 23 00:51:33.723201 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.723011    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2" event={"ID":"fe307395-2fd4-4977-afeb-1105339adedc","Type":"ContainerDied","Data":"b693307ac561fae42ae19bf5c9e591076b69e5ec05933324eecc9f82149c1c46"}
Jul 23 00:51:33.723201 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.723037    2561 scope.go:117] "RemoveContainer" containerID="c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76"
Jul 23 00:51:33.733331 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.733315    2561 scope.go:117] "RemoveContainer" containerID="c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76"
Jul 23 00:51:33.733652 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:51:33.733629    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76\": container with ID starting with c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76 not found: ID does not exist" containerID="c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76"
Jul 23 00:51:33.733710 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.733663    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76"} err="failed to get container status \"c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76\": rpc error: code = NotFound desc = could not find container \"c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76\": container with ID starting with c255054e26fd71b4caa06da575da6ae9ed155d9fcd8c3bf5edab7db4ed14ea76 not found: ID does not exist"
Jul 23 00:51:33.743751 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.743729    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"]
Jul 23 00:51:33.746973 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:33.746947    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-c49d048f-kserve-746ffb4c67-hl2g2"]
Jul 23 00:51:34.786958 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:51:34.786919    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe307395-2fd4-4977-afeb-1105339adedc" path="/var/lib/kubelet/pods/fe307395-2fd4-4977-afeb-1105339adedc/volumes"
Jul 23 00:53:16.780100 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:53:16.780073    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:53:16.782385 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:53:16.782343    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:55:00.976430 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:00.976395    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"]
Jul 23 00:55:00.976968 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:00.976847    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe307395-2fd4-4977-afeb-1105339adedc" containerName="main"
Jul 23 00:55:00.976968 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:00.976863    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe307395-2fd4-4977-afeb-1105339adedc" containerName="main"
Jul 23 00:55:00.976968 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:00.976933    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe307395-2fd4-4977-afeb-1105339adedc" containerName="main"
Jul 23 00:55:00.979315 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:00.979299    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:00.982043 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:00.982019    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5327912a-openshift-default-dockercfg-fxcdb\""
Jul 23 00:55:01.017290 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.017263    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"]
Jul 23 00:55:01.066389 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.066343    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-credential-socket\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.066389 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.066394    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-podinfo\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.066698 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.066414    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrhjr\" (UniqueName: \"kubernetes.io/projected/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-kube-api-access-xrhjr\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.066698 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.066545    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istiod-ca-cert\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.066698 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.066589    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-workload-socket\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.066698 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.066618    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-token\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.066698 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.066659    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-workload-certs\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.066919 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.066740    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-envoy\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.066919 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.066763    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-data\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.167734 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.167699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrhjr\" (UniqueName: \"kubernetes.io/projected/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-kube-api-access-xrhjr\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.167904 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.167745    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istiod-ca-cert\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.167904 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.167768    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-workload-socket\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.167904 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.167795    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-token\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.167904 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.167813    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-workload-certs\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.167904 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.167834    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-envoy\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.167904 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.167852    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-data\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.168228 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.167926    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-credential-socket\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.168228 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.167961    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-podinfo\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.168322 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.168222    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-workload-socket\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.168322 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.168304    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-workload-certs\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.168455 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.168391    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-credential-socket\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.168455 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.168447    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-data\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.168559 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.168502    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istiod-ca-cert\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.170190 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.170168    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-envoy\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.170325 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.170309    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-podinfo\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.176704 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.176680    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-token\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.176704 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.176697    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrhjr\" (UniqueName: \"kubernetes.io/projected/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-kube-api-access-xrhjr\") pod \"e2e-ait-5327912a-openshift-default-79b48897f-jcdp6\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") " pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.290427 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.290331    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:01.424315 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.424290    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"]
Jul 23 00:55:01.426720 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:55:01.426695    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4217d5a_a4b4_4fdd_98c8_a3e06af04fcb.slice/crio-a761266738bfb9fab00bb716a234af0380b88e834b421825677825bbdfb28a24 WatchSource:0}: Error finding container a761266738bfb9fab00bb716a234af0380b88e834b421825677825bbdfb28a24: Status 404 returned error can't find the container with id a761266738bfb9fab00bb716a234af0380b88e834b421825677825bbdfb28a24
Jul 23 00:55:01.429510 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.429490    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 00:55:01.661761 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:01.661656    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6" event={"ID":"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb","Type":"ContainerStarted","Data":"a761266738bfb9fab00bb716a234af0380b88e834b421825677825bbdfb28a24"}
Jul 23 00:55:05.077743 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:05.077708    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:55:05.078116 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:05.077798    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:55:05.078116 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:05.077846    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:55:05.685757 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:05.685718    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6" event={"ID":"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb","Type":"ContainerStarted","Data":"8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162"}
Jul 23 00:55:05.708011 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:05.707950    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6" podStartSLOduration=2.060201331 podStartE2EDuration="5.707935998s" totalImagesPullingTime="3.647734667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:55:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:55:05.705731204 +0000 UTC m=+1909.512142475" watchObservedRunningTime="2026-07-23 00:55:05.707935998 +0000 UTC m=+1909.514347230"
Jul 23 00:55:06.290540 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:06.290502    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:06.295751 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:06.295725    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:06.691066 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:06.691036    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:06.692168 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:06.692150    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:07.156049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.155958    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"]
Jul 23 00:55:07.159458 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.159436    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:07.165741 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.165713    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5327912a-serving-cert\""
Jul 23 00:55:07.172828 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.172802    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"]
Jul 23 00:55:07.322761 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.322717    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qbdh\" (UniqueName: \"kubernetes.io/projected/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-kube-api-access-7qbdh\") pod \"maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx\" (UID: \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:07.323226 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.322836    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-maas-api-tls\") pod \"maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx\" (UID: \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:07.323226 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.322898    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-openshift-service-ca\") pod \"maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx\" (UID: \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:07.423678 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.423585    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-maas-api-tls\") pod \"maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx\" (UID: \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:07.423678 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.423630    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-openshift-service-ca\") pod \"maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx\" (UID: \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:07.423913 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.423682    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qbdh\" (UniqueName: \"kubernetes.io/projected/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-kube-api-access-7qbdh\") pod \"maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx\" (UID: \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:07.424393 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.424350    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-openshift-service-ca\") pod \"maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx\" (UID: \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:07.426514 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.426474    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-maas-api-tls\") pod \"maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx\" (UID: \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:07.431603 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.431576    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qbdh\" (UniqueName: \"kubernetes.io/projected/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-kube-api-access-7qbdh\") pod \"maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx\" (UID: \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:07.471514 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.471475    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:07.618342 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.618318    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"]
Jul 23 00:55:07.619904 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:55:07.619875    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f36e7e7_5a8f_43df_a0bc_6c8f1d386780.slice/crio-8fd05489939b194986e8b46eb443a2a4a3bed8fc8a968374e4ebe594e9a47f7b WatchSource:0}: Error finding container 8fd05489939b194986e8b46eb443a2a4a3bed8fc8a968374e4ebe594e9a47f7b: Status 404 returned error can't find the container with id 8fd05489939b194986e8b46eb443a2a4a3bed8fc8a968374e4ebe594e9a47f7b
Jul 23 00:55:07.696269 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:07.696227    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx" event={"ID":"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780","Type":"ContainerStarted","Data":"8fd05489939b194986e8b46eb443a2a4a3bed8fc8a968374e4ebe594e9a47f7b"}
Jul 23 00:55:08.702244 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:08.702207    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx" event={"ID":"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780","Type":"ContainerStarted","Data":"10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366"}
Jul 23 00:55:08.702766 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:08.702301    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:08.722055 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:08.722004    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx" podStartSLOduration=1.195827196 podStartE2EDuration="1.721987513s" totalImagesPullingTime="526.160317ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:55:08.720138743 +0000 UTC m=+1912.526549988" watchObservedRunningTime="2026-07-23 00:55:08.721987513 +0000 UTC m=+1912.528398745"
Jul 23 00:55:14.712383 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:14.712341    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:19.763745 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:19.763707    2561 generic.go:356] "Generic (PLEG): container finished" podID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerID="8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162" exitCode=0
Jul 23 00:55:19.764173 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:19.763776    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6" event={"ID":"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb","Type":"ContainerDied","Data":"8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162"}
Jul 23 00:55:19.764173 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:19.764153    2561 scope.go:117] "RemoveContainer" containerID="8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162"
Jul 23 00:55:19.765066 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:19.765027    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:55:19.765176 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:19.765112    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:55:19.765176 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:19.765159    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:55:20.770356 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:20.770322    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6" event={"ID":"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb","Type":"ContainerStarted","Data":"60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4"}
Jul 23 00:55:21.290610 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:21.290575    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:21.291884 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:21.291859    2561 patch_prober.go:28] interesting pod/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Jul 23 00:55:21.291967 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:21.291928    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6" podUID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Jul 23 00:55:21.691711 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:21.691676    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:22.291408 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:22.291348    2561 patch_prober.go:28] interesting pod/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Jul 23 00:55:22.291813 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:22.291432    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6" podUID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Jul 23 00:55:23.291356 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:23.291315    2561 patch_prober.go:28] interesting pod/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Jul 23 00:55:23.291554 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:23.291406    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6" podUID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Jul 23 00:55:24.290795 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:24.290756    2561 patch_prober.go:28] interesting pod/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Jul 23 00:55:24.290994 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:24.290813    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6" podUID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Jul 23 00:55:25.294859 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:25.294827    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:25.295877 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:25.295855    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:43.066898 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.066861    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"]
Jul 23 00:55:43.067527 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.067156    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx" podUID="6f36e7e7-5a8f-43df-a0bc-6c8f1d386780" containerName="maas-api" containerID="cri-o://10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366" gracePeriod=30
Jul 23 00:55:43.316313 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.316287    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:43.447289 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.447258    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qbdh\" (UniqueName: \"kubernetes.io/projected/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-kube-api-access-7qbdh\") pod \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\" (UID: \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\") "
Jul 23 00:55:43.447511 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.447334    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-openshift-service-ca\") pod \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\" (UID: \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\") "
Jul 23 00:55:43.447511 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.447405    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-maas-api-tls\") pod \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\" (UID: \"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780\") "
Jul 23 00:55:43.447715 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.447687    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f36e7e7-5a8f-43df-a0bc-6c8f1d386780" (UID: "6f36e7e7-5a8f-43df-a0bc-6c8f1d386780"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:55:43.449626 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.449603    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f36e7e7-5a8f-43df-a0bc-6c8f1d386780" (UID: "6f36e7e7-5a8f-43df-a0bc-6c8f1d386780"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:55:43.449704 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.449625    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-kube-api-access-7qbdh" (OuterVolumeSpecName: "kube-api-access-7qbdh") pod "6f36e7e7-5a8f-43df-a0bc-6c8f1d386780" (UID: "6f36e7e7-5a8f-43df-a0bc-6c8f1d386780"). InnerVolumeSpecName "kube-api-access-7qbdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:55:43.548336 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.548300    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-openshift-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:55:43.548336 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.548331    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-maas-api-tls\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:55:43.548336 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.548344    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qbdh\" (UniqueName: \"kubernetes.io/projected/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780-kube-api-access-7qbdh\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:55:43.874001 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.873905    2561 generic.go:356] "Generic (PLEG): container finished" podID="6f36e7e7-5a8f-43df-a0bc-6c8f1d386780" containerID="10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366" exitCode=0
Jul 23 00:55:43.874001 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.873945    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"
Jul 23 00:55:43.874001 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.873990    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx" event={"ID":"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780","Type":"ContainerDied","Data":"10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366"}
Jul 23 00:55:43.874252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.874038    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx" event={"ID":"6f36e7e7-5a8f-43df-a0bc-6c8f1d386780","Type":"ContainerDied","Data":"8fd05489939b194986e8b46eb443a2a4a3bed8fc8a968374e4ebe594e9a47f7b"}
Jul 23 00:55:43.874252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.874061    2561 scope.go:117] "RemoveContainer" containerID="10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366"
Jul 23 00:55:43.882997 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.882980    2561 scope.go:117] "RemoveContainer" containerID="10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366"
Jul 23 00:55:43.883243 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:55:43.883225    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366\": container with ID starting with 10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366 not found: ID does not exist" containerID="10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366"
Jul 23 00:55:43.883296 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.883251    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366"} err="failed to get container status \"10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366\": rpc error: code = NotFound desc = could not find container \"10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366\": container with ID starting with 10f78b30f148d0c177706d9c64d54d6dd65430278982cb340f9f23162623f366 not found: ID does not exist"
Jul 23 00:55:43.894887 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.894863    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"]
Jul 23 00:55:43.898874 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:43.898853    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5327912a-6c4fc9c674-47kfx"]
Jul 23 00:55:44.786865 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:44.786821    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f36e7e7-5a8f-43df-a0bc-6c8f1d386780" path="/var/lib/kubelet/pods/6f36e7e7-5a8f-43df-a0bc-6c8f1d386780/volumes"
Jul 23 00:55:47.843759 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:47.843719    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"]
Jul 23 00:55:47.844245 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:47.844027    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6" podUID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerName="istio-proxy" containerID="cri-o://60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4" gracePeriod=30
Jul 23 00:55:48.872753 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.872716    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"]
Jul 23 00:55:48.873147 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.873049    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f36e7e7-5a8f-43df-a0bc-6c8f1d386780" containerName="maas-api"
Jul 23 00:55:48.873147 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.873059    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f36e7e7-5a8f-43df-a0bc-6c8f1d386780" containerName="maas-api"
Jul 23 00:55:48.873147 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.873130    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f36e7e7-5a8f-43df-a0bc-6c8f1d386780" containerName="maas-api"
Jul 23 00:55:48.875959 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.875938    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:48.878544 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.878525    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-5d1adec3-gw-openshift-default-dockercfg-jbvlr\""
Jul 23 00:55:48.887753 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.887728    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"]
Jul 23 00:55:48.991135 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.991103    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-credential-socket\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:48.991330 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.991211    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-envoy\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:48.991330 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.991274    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istiod-ca-cert\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:48.991466 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.991344    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-token\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:48.991466 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.991400    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-workload-socket\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:48.991466 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.991446    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-workload-certs\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:48.991573 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.991477    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-data\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:48.991573 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.991557    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-podinfo\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:48.991639 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:48.991575    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbtxr\" (UniqueName: \"kubernetes.io/projected/ed23d63c-8965-4cb6-aeb3-c94175609cc0-kube-api-access-vbtxr\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.091999 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.091959    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-credential-socket\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.091999 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092001    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-envoy\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.092261 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092021    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istiod-ca-cert\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.092261 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092153    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-token\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.092261 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092201    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-workload-socket\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.092261 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092250    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-workload-certs\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.092511 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092273    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-data\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.092511 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092488    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-credential-socket\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.092647 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092614    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-workload-socket\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.092720 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092686    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-data\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.092720 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092706    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-workload-certs\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.092830 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092359    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-podinfo\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.092830 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092774    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbtxr\" (UniqueName: \"kubernetes.io/projected/ed23d63c-8965-4cb6-aeb3-c94175609cc0-kube-api-access-vbtxr\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.092830 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.092784    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istiod-ca-cert\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.094468 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.094448    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-envoy\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.094809 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.094793    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-podinfo\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.100474 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.100450    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-token\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.100870 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.100829    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbtxr\" (UniqueName: \"kubernetes.io/projected/ed23d63c-8965-4cb6-aeb3-c94175609cc0-kube-api-access-vbtxr\") pod \"e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") " pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.185227 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.185196    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:49.325210 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.325185    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"]
Jul 23 00:55:49.327258 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:55:49.327228    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded23d63c_8965_4cb6_aeb3_c94175609cc0.slice/crio-9c52b4834e949ab54787be593c5043ebd931c2d376fbf4a649e0a677c27685cc WatchSource:0}: Error finding container 9c52b4834e949ab54787be593c5043ebd931c2d376fbf4a649e0a677c27685cc: Status 404 returned error can't find the container with id 9c52b4834e949ab54787be593c5043ebd931c2d376fbf4a649e0a677c27685cc
Jul 23 00:55:49.329618 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.329587    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:55:49.329697 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.329662    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:55:49.329697 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.329693    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:55:49.901784 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.901747    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" event={"ID":"ed23d63c-8965-4cb6-aeb3-c94175609cc0","Type":"ContainerStarted","Data":"d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4"}
Jul 23 00:55:49.902167 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.901805    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" event={"ID":"ed23d63c-8965-4cb6-aeb3-c94175609cc0","Type":"ContainerStarted","Data":"9c52b4834e949ab54787be593c5043ebd931c2d376fbf4a649e0a677c27685cc"}
Jul 23 00:55:49.925458 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:49.925411    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" podStartSLOduration=1.925395799 podStartE2EDuration="1.925395799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:55:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:55:49.923054526 +0000 UTC m=+1953.729465798" watchObservedRunningTime="2026-07-23 00:55:49.925395799 +0000 UTC m=+1953.731807104"
Jul 23 00:55:50.186355 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:50.186323    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:50.191008 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:50.190985    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:50.905829 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:50.905803    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:50.906820 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:50.906797    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:55:51.639115 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.639084    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"]
Jul 23 00:55:51.642515 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.642496    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:55:51.644916 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.644892    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-5d1adec3-serving-cert\""
Jul 23 00:55:51.653645 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.653620    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"]
Jul 23 00:55:51.717560 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.717526    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nqw8\" (UniqueName: \"kubernetes.io/projected/16af761e-beb2-4240-9a74-0bc672d3cb4d-kube-api-access-2nqw8\") pod \"maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs\" (UID: \"16af761e-beb2-4240-9a74-0bc672d3cb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:55:51.717696 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.717583    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16af761e-beb2-4240-9a74-0bc672d3cb4d-openshift-service-ca\") pod \"maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs\" (UID: \"16af761e-beb2-4240-9a74-0bc672d3cb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:55:51.717696 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.717668    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16af761e-beb2-4240-9a74-0bc672d3cb4d-maas-api-tls\") pod \"maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs\" (UID: \"16af761e-beb2-4240-9a74-0bc672d3cb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:55:51.818988 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.818946    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nqw8\" (UniqueName: \"kubernetes.io/projected/16af761e-beb2-4240-9a74-0bc672d3cb4d-kube-api-access-2nqw8\") pod \"maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs\" (UID: \"16af761e-beb2-4240-9a74-0bc672d3cb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:55:51.819160 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.819020    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16af761e-beb2-4240-9a74-0bc672d3cb4d-openshift-service-ca\") pod \"maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs\" (UID: \"16af761e-beb2-4240-9a74-0bc672d3cb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:55:51.819160 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.819058    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16af761e-beb2-4240-9a74-0bc672d3cb4d-maas-api-tls\") pod \"maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs\" (UID: \"16af761e-beb2-4240-9a74-0bc672d3cb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:55:51.819669 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.819649    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16af761e-beb2-4240-9a74-0bc672d3cb4d-openshift-service-ca\") pod \"maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs\" (UID: \"16af761e-beb2-4240-9a74-0bc672d3cb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:55:51.821439 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.821424    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16af761e-beb2-4240-9a74-0bc672d3cb4d-maas-api-tls\") pod \"maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs\" (UID: \"16af761e-beb2-4240-9a74-0bc672d3cb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:55:51.829185 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.829159    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nqw8\" (UniqueName: \"kubernetes.io/projected/16af761e-beb2-4240-9a74-0bc672d3cb4d-kube-api-access-2nqw8\") pod \"maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs\" (UID: \"16af761e-beb2-4240-9a74-0bc672d3cb4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:55:51.956194 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:51.956160    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:55:52.091037 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:52.091015    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"]
Jul 23 00:55:52.093274 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:55:52.093247    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16af761e_beb2_4240_9a74_0bc672d3cb4d.slice/crio-756146c86ab30278fdf2f7dee0b5436e6c35779a377366e675d10a92327fc970 WatchSource:0}: Error finding container 756146c86ab30278fdf2f7dee0b5436e6c35779a377366e675d10a92327fc970: Status 404 returned error can't find the container with id 756146c86ab30278fdf2f7dee0b5436e6c35779a377366e675d10a92327fc970
Jul 23 00:55:52.917231 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:52.917191    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs" event={"ID":"16af761e-beb2-4240-9a74-0bc672d3cb4d","Type":"ContainerStarted","Data":"1397468678cd41651d15f29bf8aa5a986e1a59e65ae65d3bc44ba13dfe53030d"}
Jul 23 00:55:52.917551 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:52.917528    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs" event={"ID":"16af761e-beb2-4240-9a74-0bc672d3cb4d","Type":"ContainerStarted","Data":"756146c86ab30278fdf2f7dee0b5436e6c35779a377366e675d10a92327fc970"}
Jul 23 00:55:52.917704 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:52.917691    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:55:52.936117 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:52.936068    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs" podStartSLOduration=1.573349593 podStartE2EDuration="1.936053591s" totalImagesPullingTime="362.703998ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:55:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:55:52.934612371 +0000 UTC m=+1956.741023618" watchObservedRunningTime="2026-07-23 00:55:52.936053591 +0000 UTC m=+1956.742464824"
Jul 23 00:55:53.102620 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.102593    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:53.231957 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.231921    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrhjr\" (UniqueName: \"kubernetes.io/projected/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-kube-api-access-xrhjr\") pod \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") "
Jul 23 00:55:53.232141 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232007    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-podinfo\") pod \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") "
Jul 23 00:55:53.232141 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232049    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-credential-socket\") pod \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") "
Jul 23 00:55:53.232141 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232076    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istiod-ca-cert\") pod \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") "
Jul 23 00:55:53.232141 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232101    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-workload-certs\") pod \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") "
Jul 23 00:55:53.232141 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232131    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-token\") pod \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") "
Jul 23 00:55:53.232443 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232169    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-data\") pod \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") "
Jul 23 00:55:53.232443 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232204    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-workload-socket\") pod \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") "
Jul 23 00:55:53.232443 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232254    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-envoy\") pod \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\" (UID: \"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb\") "
Jul 23 00:55:53.232609 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232430    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" (UID: "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:55:53.232665 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232647    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" (UID: "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:55:53.232799 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232740    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:55:53.232799 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232767    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:55:53.233005 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232880    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" (UID: "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:55:53.233005 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.232903    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" (UID: "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:55:53.233316 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.233292    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-data" (OuterVolumeSpecName: "istio-data") pod "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" (UID: "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:55:53.234871 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.234843    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-kube-api-access-xrhjr" (OuterVolumeSpecName: "kube-api-access-xrhjr") pod "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" (UID: "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb"). InnerVolumeSpecName "kube-api-access-xrhjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:55:53.234995 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.234884    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" (UID: "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:55:53.235141 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.235122    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" (UID: "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:55:53.235406 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.235386    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" (UID: "a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 00:55:53.334126 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.334085    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:55:53.334126 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.334114    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:55:53.334126 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.334124    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:55:53.334126 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.334131    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:55:53.334126 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.334140    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:55:53.334540 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.334148    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:55:53.334540 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.334157    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrhjr\" (UniqueName: \"kubernetes.io/projected/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb-kube-api-access-xrhjr\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:55:53.921823 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.921789    2561 generic.go:356] "Generic (PLEG): container finished" podID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerID="60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4" exitCode=0
Jul 23 00:55:53.922003 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.921844    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"
Jul 23 00:55:53.922003 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.921876    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6" event={"ID":"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb","Type":"ContainerDied","Data":"60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4"}
Jul 23 00:55:53.922003 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.921932    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6" event={"ID":"a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb","Type":"ContainerDied","Data":"a761266738bfb9fab00bb716a234af0380b88e834b421825677825bbdfb28a24"}
Jul 23 00:55:53.922003 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.921953    2561 scope.go:117] "RemoveContainer" containerID="60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4"
Jul 23 00:55:53.931846 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.931823    2561 scope.go:117] "RemoveContainer" containerID="8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162"
Jul 23 00:55:53.940252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.940227    2561 scope.go:117] "RemoveContainer" containerID="60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4"
Jul 23 00:55:53.940865 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:55:53.940843    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4\": container with ID starting with 60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4 not found: ID does not exist" containerID="60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4"
Jul 23 00:55:53.940940 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.940877    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4"} err="failed to get container status \"60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4\": rpc error: code = NotFound desc = could not find container \"60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4\": container with ID starting with 60457fc4b217b0cf54b0201ce14ffe4981bf744f2ee96cd48e45250aa223c2c4 not found: ID does not exist"
Jul 23 00:55:53.940940 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.940897    2561 scope.go:117] "RemoveContainer" containerID="8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162"
Jul 23 00:55:53.941161 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:55:53.941137    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162\": container with ID starting with 8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162 not found: ID does not exist" containerID="8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162"
Jul 23 00:55:53.941203 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.941173    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162"} err="failed to get container status \"8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162\": rpc error: code = NotFound desc = could not find container \"8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162\": container with ID starting with 8ec9d6cc56586fc72b37c1692e2d0f06291ede613d1e7b75aa6d15488ec9b162 not found: ID does not exist"
Jul 23 00:55:53.945103 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.945077    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"]
Jul 23 00:55:53.953917 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:53.953891    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5327912a-openshift-default-79b48897f-jcdp6"]
Jul 23 00:55:54.787645 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:54.787610    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" path="/var/lib/kubelet/pods/a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb/volumes"
Jul 23 00:55:59.942742 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:55:59.942700    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:56:06.346831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:06.346797    2561 patch_prober.go:28] interesting pod/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:50148->10.132.0.40:15021: read: connection reset by peer" start-of-body=
Jul 23 00:56:06.347317 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:06.346869    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:50148->10.132.0.40:15021: read: connection reset by peer"
Jul 23 00:56:06.988320 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:06.988287    2561 generic.go:356] "Generic (PLEG): container finished" podID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerID="d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4" exitCode=0
Jul 23 00:56:06.988508 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:06.988375    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" event={"ID":"ed23d63c-8965-4cb6-aeb3-c94175609cc0","Type":"ContainerDied","Data":"d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4"}
Jul 23 00:56:06.988823 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:06.988805    2561 scope.go:117] "RemoveContainer" containerID="d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4"
Jul 23 00:56:06.989856 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:06.989831    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:56:06.989916 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:06.989896    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:56:06.989954 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:06.989929    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:56:07.995282 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:07.995205    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" event={"ID":"ed23d63c-8965-4cb6-aeb3-c94175609cc0","Type":"ContainerStarted","Data":"ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b"}
Jul 23 00:56:08.186190 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:08.186149    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:56:08.186450 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:08.186428    2561 patch_prober.go:28] interesting pod/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 23 00:56:08.186502 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:08.186479    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 23 00:56:09.186246 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:09.186212    2561 patch_prober.go:28] interesting pod/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 23 00:56:09.186695 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:09.186286    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 23 00:56:10.186562 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:10.186521    2561 patch_prober.go:28] interesting pod/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 23 00:56:10.186980 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:10.186581    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 23 00:56:11.186034 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:11.185996    2561 patch_prober.go:28] interesting pod/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 23 00:56:11.186199 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:11.186054    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 23 00:56:12.189894 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:12.189863    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:56:12.190396 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:12.190172    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:56:12.190952 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:12.190937    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:56:26.873488 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:26.873453    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"]
Jul 23 00:56:26.874004 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:26.873646    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs" podUID="16af761e-beb2-4240-9a74-0bc672d3cb4d" containerName="maas-api" containerID="cri-o://1397468678cd41651d15f29bf8aa5a986e1a59e65ae65d3bc44ba13dfe53030d" gracePeriod=30
Jul 23 00:56:27.084168 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:27.084136    2561 generic.go:356] "Generic (PLEG): container finished" podID="16af761e-beb2-4240-9a74-0bc672d3cb4d" containerID="1397468678cd41651d15f29bf8aa5a986e1a59e65ae65d3bc44ba13dfe53030d" exitCode=0
Jul 23 00:56:27.084320 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:27.084209    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs" event={"ID":"16af761e-beb2-4240-9a74-0bc672d3cb4d","Type":"ContainerDied","Data":"1397468678cd41651d15f29bf8aa5a986e1a59e65ae65d3bc44ba13dfe53030d"}
Jul 23 00:56:27.125300 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:27.125244    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:56:27.218608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:27.218580    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16af761e-beb2-4240-9a74-0bc672d3cb4d-openshift-service-ca\") pod \"16af761e-beb2-4240-9a74-0bc672d3cb4d\" (UID: \"16af761e-beb2-4240-9a74-0bc672d3cb4d\") "
Jul 23 00:56:27.218608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:27.218613    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16af761e-beb2-4240-9a74-0bc672d3cb4d-maas-api-tls\") pod \"16af761e-beb2-4240-9a74-0bc672d3cb4d\" (UID: \"16af761e-beb2-4240-9a74-0bc672d3cb4d\") "
Jul 23 00:56:27.218865 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:27.218678    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nqw8\" (UniqueName: \"kubernetes.io/projected/16af761e-beb2-4240-9a74-0bc672d3cb4d-kube-api-access-2nqw8\") pod \"16af761e-beb2-4240-9a74-0bc672d3cb4d\" (UID: \"16af761e-beb2-4240-9a74-0bc672d3cb4d\") "
Jul 23 00:56:27.219007 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:27.218981    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16af761e-beb2-4240-9a74-0bc672d3cb4d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16af761e-beb2-4240-9a74-0bc672d3cb4d" (UID: "16af761e-beb2-4240-9a74-0bc672d3cb4d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:56:27.220751 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:27.220732    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16af761e-beb2-4240-9a74-0bc672d3cb4d-kube-api-access-2nqw8" (OuterVolumeSpecName: "kube-api-access-2nqw8") pod "16af761e-beb2-4240-9a74-0bc672d3cb4d" (UID: "16af761e-beb2-4240-9a74-0bc672d3cb4d"). InnerVolumeSpecName "kube-api-access-2nqw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:56:27.220855 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:27.220834    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16af761e-beb2-4240-9a74-0bc672d3cb4d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16af761e-beb2-4240-9a74-0bc672d3cb4d" (UID: "16af761e-beb2-4240-9a74-0bc672d3cb4d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:56:27.319461 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:27.319422    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16af761e-beb2-4240-9a74-0bc672d3cb4d-maas-api-tls\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:56:27.319461 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:27.319455    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nqw8\" (UniqueName: \"kubernetes.io/projected/16af761e-beb2-4240-9a74-0bc672d3cb4d-kube-api-access-2nqw8\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:56:27.319461 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:27.319468    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16af761e-beb2-4240-9a74-0bc672d3cb4d-openshift-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:56:28.089630 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:28.089599    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"
Jul 23 00:56:28.089630 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:28.089627    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs" event={"ID":"16af761e-beb2-4240-9a74-0bc672d3cb4d","Type":"ContainerDied","Data":"756146c86ab30278fdf2f7dee0b5436e6c35779a377366e675d10a92327fc970"}
Jul 23 00:56:28.090192 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:28.089680    2561 scope.go:117] "RemoveContainer" containerID="1397468678cd41651d15f29bf8aa5a986e1a59e65ae65d3bc44ba13dfe53030d"
Jul 23 00:56:28.110675 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:28.110649    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"]
Jul 23 00:56:28.114652 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:28.114630    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5d1adec3-6cddf94dc5-sbgfs"]
Jul 23 00:56:28.787011 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:28.786975    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16af761e-beb2-4240-9a74-0bc672d3cb4d" path="/var/lib/kubelet/pods/16af761e-beb2-4240-9a74-0bc672d3cb4d/volumes"
Jul 23 00:56:31.618282 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:31.618250    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"]
Jul 23 00:56:31.618703 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:31.618504    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerName="istio-proxy" containerID="cri-o://ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b" gracePeriod=30
Jul 23 00:56:32.394444 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.394412    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"]
Jul 23 00:56:32.394782 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.394769    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerName="istio-proxy"
Jul 23 00:56:32.394831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.394783    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerName="istio-proxy"
Jul 23 00:56:32.394831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.394810    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16af761e-beb2-4240-9a74-0bc672d3cb4d" containerName="maas-api"
Jul 23 00:56:32.394831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.394816    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="16af761e-beb2-4240-9a74-0bc672d3cb4d" containerName="maas-api"
Jul 23 00:56:32.394947 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.394903    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerName="istio-proxy"
Jul 23 00:56:32.394947 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.394914    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="16af761e-beb2-4240-9a74-0bc672d3cb4d" containerName="maas-api"
Jul 23 00:56:32.394947 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.394928    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerName="istio-proxy"
Jul 23 00:56:32.397831 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.397806    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.400588 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.400568    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-882c377a-openshift-default-dockercfg-82nvs\""
Jul 23 00:56:32.409984 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.409960    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"]
Jul 23 00:56:32.467835 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.467796    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j69sq\" (UniqueName: \"kubernetes.io/projected/cad80b61-7c72-479d-9fc4-5c2fe5441474-kube-api-access-j69sq\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.468019 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.467847    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cad80b61-7c72-479d-9fc4-5c2fe5441474-istiod-ca-cert\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.468019 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.467913    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-credential-socket\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.468019 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.467953    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-podinfo\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.468019 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.467990    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-workload-certs\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.468019 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.468008    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-token\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.468223 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.468028    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-workload-socket\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.468223 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.468070    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-envoy\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.468223 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.468114    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-data\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.569381 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.569328    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-workload-certs\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.569564 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.569392    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-token\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.569564 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.569425    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-workload-socket\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.569564 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.569470    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-envoy\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.569564 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.569505    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-data\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.569564 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.569555    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j69sq\" (UniqueName: \"kubernetes.io/projected/cad80b61-7c72-479d-9fc4-5c2fe5441474-kube-api-access-j69sq\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.569852 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.569600    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cad80b61-7c72-479d-9fc4-5c2fe5441474-istiod-ca-cert\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.569852 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.569631    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-credential-socket\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.569852 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.569660    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-podinfo\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.569852 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.569778    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-workload-certs\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.569852 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.569814    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-workload-socket\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.570120 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.569878    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-data\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.570120 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.570071    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-credential-socket\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.570385 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.570350    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cad80b61-7c72-479d-9fc4-5c2fe5441474-istiod-ca-cert\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.571803 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.571784    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-envoy\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.572069 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.572050    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-podinfo\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.577144 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.577099    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j69sq\" (UniqueName: \"kubernetes.io/projected/cad80b61-7c72-479d-9fc4-5c2fe5441474-kube-api-access-j69sq\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.577234 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.577225    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-token\") pod \"e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") " pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.707713 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.707675    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:32.844441 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.844407    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"]
Jul 23 00:56:32.846674 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:56:32.846645    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad80b61_7c72_479d_9fc4_5c2fe5441474.slice/crio-dbcc66077aefb1095310ab81b356c7138d09acb6f24244db977d92f838d42db0 WatchSource:0}: Error finding container dbcc66077aefb1095310ab81b356c7138d09acb6f24244db977d92f838d42db0: Status 404 returned error can't find the container with id dbcc66077aefb1095310ab81b356c7138d09acb6f24244db977d92f838d42db0
Jul 23 00:56:32.849047 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.849008    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:56:32.849152 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.849088    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:56:32.849152 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:32.849130    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:56:33.113797 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:33.113709    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" event={"ID":"cad80b61-7c72-479d-9fc4-5c2fe5441474","Type":"ContainerStarted","Data":"fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6"}
Jul 23 00:56:33.113797 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:33.113762    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" event={"ID":"cad80b61-7c72-479d-9fc4-5c2fe5441474","Type":"ContainerStarted","Data":"dbcc66077aefb1095310ab81b356c7138d09acb6f24244db977d92f838d42db0"}
Jul 23 00:56:33.140902 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:33.140833    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" podStartSLOduration=1.140779201 podStartE2EDuration="1.140779201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:56:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:56:33.139002468 +0000 UTC m=+1996.945413702" watchObservedRunningTime="2026-07-23 00:56:33.140779201 +0000 UTC m=+1996.947190439"
Jul 23 00:56:33.708473 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:33.708432    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:33.713659 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:33.713632    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:34.118693 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:34.118606    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:34.119583 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:34.119561    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:36.879345 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:36.879285    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:56:37.008252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008219    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-envoy\") pod \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") "
Jul 23 00:56:37.008454 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008273    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-podinfo\") pod \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") "
Jul 23 00:56:37.008454 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008305    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-data\") pod \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") "
Jul 23 00:56:37.008454 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008328    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-workload-certs\") pod \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") "
Jul 23 00:56:37.008454 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008346    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbtxr\" (UniqueName: \"kubernetes.io/projected/ed23d63c-8965-4cb6-aeb3-c94175609cc0-kube-api-access-vbtxr\") pod \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") "
Jul 23 00:56:37.008454 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008411    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-token\") pod \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") "
Jul 23 00:56:37.008454 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008441    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-workload-socket\") pod \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") "
Jul 23 00:56:37.008769 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008474    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istiod-ca-cert\") pod \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") "
Jul 23 00:56:37.008769 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008502    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-credential-socket\") pod \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\" (UID: \"ed23d63c-8965-4cb6-aeb3-c94175609cc0\") "
Jul 23 00:56:37.008769 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008690    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed23d63c-8965-4cb6-aeb3-c94175609cc0" (UID: "ed23d63c-8965-4cb6-aeb3-c94175609cc0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:56:37.008933 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008781    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed23d63c-8965-4cb6-aeb3-c94175609cc0" (UID: "ed23d63c-8965-4cb6-aeb3-c94175609cc0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:56:37.008933 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008847    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:56:37.008933 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.008882    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed23d63c-8965-4cb6-aeb3-c94175609cc0" (UID: "ed23d63c-8965-4cb6-aeb3-c94175609cc0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:56:37.009081 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.009062    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed23d63c-8965-4cb6-aeb3-c94175609cc0" (UID: "ed23d63c-8965-4cb6-aeb3-c94175609cc0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:56:37.009200 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.009179    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed23d63c-8965-4cb6-aeb3-c94175609cc0" (UID: "ed23d63c-8965-4cb6-aeb3-c94175609cc0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:56:37.010684 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.010650    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed23d63c-8965-4cb6-aeb3-c94175609cc0" (UID: "ed23d63c-8965-4cb6-aeb3-c94175609cc0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:56:37.010804 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.010725    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed23d63c-8965-4cb6-aeb3-c94175609cc0-kube-api-access-vbtxr" (OuterVolumeSpecName: "kube-api-access-vbtxr") pod "ed23d63c-8965-4cb6-aeb3-c94175609cc0" (UID: "ed23d63c-8965-4cb6-aeb3-c94175609cc0"). InnerVolumeSpecName "kube-api-access-vbtxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:56:37.010903 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.010890    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed23d63c-8965-4cb6-aeb3-c94175609cc0" (UID: "ed23d63c-8965-4cb6-aeb3-c94175609cc0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:56:37.011000 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.010982    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed23d63c-8965-4cb6-aeb3-c94175609cc0" (UID: "ed23d63c-8965-4cb6-aeb3-c94175609cc0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 00:56:37.110034 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.110001    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:56:37.110034 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.110026    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:56:37.110034 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.110034    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:56:37.110034 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.110043    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbtxr\" (UniqueName: \"kubernetes.io/projected/ed23d63c-8965-4cb6-aeb3-c94175609cc0-kube-api-access-vbtxr\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:56:37.110542 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.110052    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:56:37.110542 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.110060    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:56:37.110542 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.110069    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed23d63c-8965-4cb6-aeb3-c94175609cc0-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:56:37.110542 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.110079    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed23d63c-8965-4cb6-aeb3-c94175609cc0-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:56:37.135596 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.135507    2561 generic.go:356] "Generic (PLEG): container finished" podID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerID="ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b" exitCode=0
Jul 23 00:56:37.135596 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.135549    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"
Jul 23 00:56:37.135826 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.135590    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" event={"ID":"ed23d63c-8965-4cb6-aeb3-c94175609cc0","Type":"ContainerDied","Data":"ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b"}
Jul 23 00:56:37.135826 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.135655    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd" event={"ID":"ed23d63c-8965-4cb6-aeb3-c94175609cc0","Type":"ContainerDied","Data":"9c52b4834e949ab54787be593c5043ebd931c2d376fbf4a649e0a677c27685cc"}
Jul 23 00:56:37.135826 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.135685    2561 scope.go:117] "RemoveContainer" containerID="ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b"
Jul 23 00:56:37.145274 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.145257    2561 scope.go:117] "RemoveContainer" containerID="d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4"
Jul 23 00:56:37.152916 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.152901    2561 scope.go:117] "RemoveContainer" containerID="ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b"
Jul 23 00:56:37.153154 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:56:37.153135    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b\": container with ID starting with ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b not found: ID does not exist" containerID="ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b"
Jul 23 00:56:37.153202 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.153164    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b"} err="failed to get container status \"ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b\": rpc error: code = NotFound desc = could not find container \"ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b\": container with ID starting with ce36bedcbc97813ab2fcb3c8ff8a399f7a61df913a164bce5c4aba46fb4d020b not found: ID does not exist"
Jul 23 00:56:37.153202 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.153182    2561 scope.go:117] "RemoveContainer" containerID="d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4"
Jul 23 00:56:37.153405 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:56:37.153385    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4\": container with ID starting with d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4 not found: ID does not exist" containerID="d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4"
Jul 23 00:56:37.153452 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.153410    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4"} err="failed to get container status \"d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4\": rpc error: code = NotFound desc = could not find container \"d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4\": container with ID starting with d0559bb1ea2d4497f3b745df7006e174bc0ea362510b7b4c3601764a996ef0d4 not found: ID does not exist"
Jul 23 00:56:37.158207 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.158186    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"]
Jul 23 00:56:37.162506 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:37.162487    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-5d1adec3-gw-openshift-default-598df64967-2vkwd"]
Jul 23 00:56:38.787100 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:38.787065    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" path="/var/lib/kubelet/pods/ed23d63c-8965-4cb6-aeb3-c94175609cc0/volumes"
Jul 23 00:56:49.748046 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:49.747950    2561 patch_prober.go:28] interesting pod/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:55162->10.132.0.42:15021: read: connection reset by peer" start-of-body=
Jul 23 00:56:49.748446 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:49.748057    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:55162->10.132.0.42:15021: read: connection reset by peer"
Jul 23 00:56:50.206475 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:50.206440    2561 generic.go:356] "Generic (PLEG): container finished" podID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerID="fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6" exitCode=0
Jul 23 00:56:50.206682 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:50.206517    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" event={"ID":"cad80b61-7c72-479d-9fc4-5c2fe5441474","Type":"ContainerDied","Data":"fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6"}
Jul 23 00:56:50.206997 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:50.206979    2561 scope.go:117] "RemoveContainer" containerID="fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6"
Jul 23 00:56:50.207993 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:50.207967    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:56:50.208085 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:50.208033    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:56:50.208085 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:50.208067    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:56:51.213192 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:51.213159    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" event={"ID":"cad80b61-7c72-479d-9fc4-5c2fe5441474","Type":"ContainerStarted","Data":"b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f"}
Jul 23 00:56:51.708715 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:51.708684    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:51.708898 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:51.708865    2561 patch_prober.go:28] interesting pod/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 23 00:56:51.708948 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:51.708913    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 23 00:56:52.708380 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:52.708339    2561 patch_prober.go:28] interesting pod/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 23 00:56:52.708824 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:52.708418    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 23 00:56:53.708663 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:53.708622    2561 patch_prober.go:28] interesting pod/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 23 00:56:53.709056 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:53.708692    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 23 00:56:54.708054 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:54.708018    2561 patch_prober.go:28] interesting pod/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 23 00:56:54.708237 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:54.708071    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 23 00:56:55.712179 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:55.712148    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:55.712642 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:55.712424    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:56:55.713091 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:56:55.713075    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:57:00.313491 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.313446    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm"]
Jul 23 00:57:00.313943 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.313779    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerName="istio-proxy"
Jul 23 00:57:00.313943 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.313790    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4217d5a-a4b4-4fdd-98c8-a3e06af04fcb" containerName="istio-proxy"
Jul 23 00:57:00.313943 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.313809    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerName="istio-proxy"
Jul 23 00:57:00.313943 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.313814    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerName="istio-proxy"
Jul 23 00:57:00.313943 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.313825    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerName="istio-proxy"
Jul 23 00:57:00.313943 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.313831    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerName="istio-proxy"
Jul 23 00:57:00.313943 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.313884    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerName="istio-proxy"
Jul 23 00:57:00.315874 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.315855    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm"
Jul 23 00:57:00.318376 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.318341    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4zd6c\""
Jul 23 00:57:00.324353 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.324327    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm"]
Jul 23 00:57:00.415409 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.415349    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65af2def-383b-47a3-89ac-fb298b7b2cce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-882c377a-8b8hm\" (UID: \"65af2def-383b-47a3-89ac-fb298b7b2cce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm"
Jul 23 00:57:00.516883 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.516840    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65af2def-383b-47a3-89ac-fb298b7b2cce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-882c377a-8b8hm\" (UID: \"65af2def-383b-47a3-89ac-fb298b7b2cce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm"
Jul 23 00:57:00.517606 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.517588    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65af2def-383b-47a3-89ac-fb298b7b2cce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-882c377a-8b8hm\" (UID: \"65af2def-383b-47a3-89ac-fb298b7b2cce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm"
Jul 23 00:57:00.627766 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.627680    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm"
Jul 23 00:57:00.778851 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:00.778826    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm"]
Jul 23 00:57:00.781180 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:57:00.781136    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65af2def_383b_47a3_89ac_fb298b7b2cce.slice/crio-2d39b39e79def4a12c7d75160f8eb7e1ddc3454e6b08983201b255a35d1fd8e8 WatchSource:0}: Error finding container 2d39b39e79def4a12c7d75160f8eb7e1ddc3454e6b08983201b255a35d1fd8e8: Status 404 returned error can't find the container with id 2d39b39e79def4a12c7d75160f8eb7e1ddc3454e6b08983201b255a35d1fd8e8
Jul 23 00:57:01.259044 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:01.258941    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm" event={"ID":"65af2def-383b-47a3-89ac-fb298b7b2cce","Type":"ContainerStarted","Data":"89606c2802227cd707012c3e8645e5b7107ec86b567636a5eadeb11c0d33a4cf"}
Jul 23 00:57:01.259044 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:01.258996    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm" event={"ID":"65af2def-383b-47a3-89ac-fb298b7b2cce","Type":"ContainerStarted","Data":"2d39b39e79def4a12c7d75160f8eb7e1ddc3454e6b08983201b255a35d1fd8e8"}
Jul 23 00:57:01.274343 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:01.274291    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm" podStartSLOduration=1.274275953 podStartE2EDuration="1.274275953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:57:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:57:01.27343134 +0000 UTC m=+2025.079842574" watchObservedRunningTime="2026-07-23 00:57:01.274275953 +0000 UTC m=+2025.080687196"
Jul 23 00:57:02.264889 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:02.264853    2561 generic.go:356] "Generic (PLEG): container finished" podID="65af2def-383b-47a3-89ac-fb298b7b2cce" containerID="89606c2802227cd707012c3e8645e5b7107ec86b567636a5eadeb11c0d33a4cf" exitCode=0
Jul 23 00:57:02.265291 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:02.264936    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm" event={"ID":"65af2def-383b-47a3-89ac-fb298b7b2cce","Type":"ContainerDied","Data":"89606c2802227cd707012c3e8645e5b7107ec86b567636a5eadeb11c0d33a4cf"}
Jul 23 00:57:03.403584 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:03.403560    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm"
Jul 23 00:57:03.437374 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:03.437330    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65af2def-383b-47a3-89ac-fb298b7b2cce-maas-api-service-ca\") pod \"65af2def-383b-47a3-89ac-fb298b7b2cce\" (UID: \"65af2def-383b-47a3-89ac-fb298b7b2cce\") "
Jul 23 00:57:03.437649 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:03.437626    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65af2def-383b-47a3-89ac-fb298b7b2cce-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "65af2def-383b-47a3-89ac-fb298b7b2cce" (UID: "65af2def-383b-47a3-89ac-fb298b7b2cce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:57:03.537936 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:03.537852    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65af2def-383b-47a3-89ac-fb298b7b2cce-maas-api-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:57:04.276492 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:04.276465    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm"
Jul 23 00:57:04.276672 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:04.276468    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm" event={"ID":"65af2def-383b-47a3-89ac-fb298b7b2cce","Type":"ContainerDied","Data":"2d39b39e79def4a12c7d75160f8eb7e1ddc3454e6b08983201b255a35d1fd8e8"}
Jul 23 00:57:04.276672 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:04.276575    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d39b39e79def4a12c7d75160f8eb7e1ddc3454e6b08983201b255a35d1fd8e8"
Jul 23 00:57:15.390566 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:15.390531    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm"]
Jul 23 00:57:15.392999 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:15.392965    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-882c377a-8b8hm"]
Jul 23 00:57:16.579150 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:16.579110    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"]
Jul 23 00:57:16.579615 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:16.579356    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerName="istio-proxy" containerID="cri-o://b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f" gracePeriod=30
Jul 23 00:57:16.789038 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:16.789004    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65af2def-383b-47a3-89ac-fb298b7b2cce" path="/var/lib/kubelet/pods/65af2def-383b-47a3-89ac-fb298b7b2cce/volumes"
Jul 23 00:57:17.355919 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.355878    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"]
Jul 23 00:57:17.356456 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.356437    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65af2def-383b-47a3-89ac-fb298b7b2cce" containerName="revoke-keys"
Jul 23 00:57:17.356456 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.356456    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="65af2def-383b-47a3-89ac-fb298b7b2cce" containerName="revoke-keys"
Jul 23 00:57:17.356624 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.356541    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="65af2def-383b-47a3-89ac-fb298b7b2cce" containerName="revoke-keys"
Jul 23 00:57:17.356624 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.356550    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed23d63c-8965-4cb6-aeb3-c94175609cc0" containerName="istio-proxy"
Jul 23 00:57:17.359688 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.359667    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.362564 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.362542    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-cc4a0bee-openshift-default-dockercfg-5m2mw\""
Jul 23 00:57:17.370236 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.370212    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"]
Jul 23 00:57:17.456752 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.456703    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-credential-socket\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.456922 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.456807    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-workload-certs\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.456922 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.456865    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm9zr\" (UniqueName: \"kubernetes.io/projected/074ef508-6034-4756-8f68-a234740cc2ae-kube-api-access-pm9zr\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.456922 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.456892    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-workload-socket\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.456922 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.456916    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-istio-data\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.457099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.456943    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/074ef508-6034-4756-8f68-a234740cc2ae-istio-token\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.457099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.457013    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-istio-envoy\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.457099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.457048    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/074ef508-6034-4756-8f68-a234740cc2ae-istiod-ca-cert\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.457099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.457068    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/074ef508-6034-4756-8f68-a234740cc2ae-istio-podinfo\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.558357 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.558321    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm9zr\" (UniqueName: \"kubernetes.io/projected/074ef508-6034-4756-8f68-a234740cc2ae-kube-api-access-pm9zr\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.558357 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.558358    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-workload-socket\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.558638 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.558399    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-istio-data\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.558638 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.558418    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/074ef508-6034-4756-8f68-a234740cc2ae-istio-token\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.558638 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.558442    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-istio-envoy\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.558638 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.558459    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/074ef508-6034-4756-8f68-a234740cc2ae-istiod-ca-cert\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.558638 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.558481    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/074ef508-6034-4756-8f68-a234740cc2ae-istio-podinfo\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.558638 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.558546    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-credential-socket\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.558638 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.558609    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-workload-certs\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.559000 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.558847    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-istio-data\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.559000 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.558960    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-workload-socket\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.559081 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.559031    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-workload-certs\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.559116 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.559073    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-credential-socket\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.559222 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.559202    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/074ef508-6034-4756-8f68-a234740cc2ae-istiod-ca-cert\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.560777 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.560754    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-istio-envoy\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.560981 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.560964    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/074ef508-6034-4756-8f68-a234740cc2ae-istio-podinfo\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.567158 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.567134    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/074ef508-6034-4756-8f68-a234740cc2ae-istio-token\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.567264 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.567246    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm9zr\" (UniqueName: \"kubernetes.io/projected/074ef508-6034-4756-8f68-a234740cc2ae-kube-api-access-pm9zr\") pod \"e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") " pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.670903 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.670784    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:17.803556 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.803527    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"]
Jul 23 00:57:17.806059 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:57:17.806031    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod074ef508_6034_4756_8f68_a234740cc2ae.slice/crio-b7893381a5f7a06c003811123d7b225f6c3778cbdf71829758ec6b9425c6512e WatchSource:0}: Error finding container b7893381a5f7a06c003811123d7b225f6c3778cbdf71829758ec6b9425c6512e: Status 404 returned error can't find the container with id b7893381a5f7a06c003811123d7b225f6c3778cbdf71829758ec6b9425c6512e
Jul 23 00:57:17.808252 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.808219    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:57:17.808334 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.808283    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:57:17.808334 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:17.808312    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:57:18.348359 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:18.348324    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" event={"ID":"074ef508-6034-4756-8f68-a234740cc2ae","Type":"ContainerStarted","Data":"4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979"}
Jul 23 00:57:18.348542 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:18.348396    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" event={"ID":"074ef508-6034-4756-8f68-a234740cc2ae","Type":"ContainerStarted","Data":"b7893381a5f7a06c003811123d7b225f6c3778cbdf71829758ec6b9425c6512e"}
Jul 23 00:57:18.369590 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:18.369547    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" podStartSLOduration=1.369533217 podStartE2EDuration="1.369533217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:57:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:57:18.367684173 +0000 UTC m=+2042.174095405" watchObservedRunningTime="2026-07-23 00:57:18.369533217 +0000 UTC m=+2042.175944450"
Jul 23 00:57:18.672035 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:18.671946    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:18.677030 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:18.677004    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:19.353114 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:19.353084    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:19.353982 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:19.353961    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:21.854756 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.854731    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:57:21.898413 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.898381    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-workload-certs\") pod \"cad80b61-7c72-479d-9fc4-5c2fe5441474\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") "
Jul 23 00:57:21.898590 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.898453    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-envoy\") pod \"cad80b61-7c72-479d-9fc4-5c2fe5441474\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") "
Jul 23 00:57:21.898590 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.898511    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-workload-socket\") pod \"cad80b61-7c72-479d-9fc4-5c2fe5441474\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") "
Jul 23 00:57:21.898590 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.898542    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cad80b61-7c72-479d-9fc4-5c2fe5441474-istiod-ca-cert\") pod \"cad80b61-7c72-479d-9fc4-5c2fe5441474\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") "
Jul 23 00:57:21.898590 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.898560    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cad80b61-7c72-479d-9fc4-5c2fe5441474" (UID: "cad80b61-7c72-479d-9fc4-5c2fe5441474"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:57:21.898590 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.898569    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-data\") pod \"cad80b61-7c72-479d-9fc4-5c2fe5441474\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") "
Jul 23 00:57:21.898864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.898626    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-token\") pod \"cad80b61-7c72-479d-9fc4-5c2fe5441474\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") "
Jul 23 00:57:21.898864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.898691    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j69sq\" (UniqueName: \"kubernetes.io/projected/cad80b61-7c72-479d-9fc4-5c2fe5441474-kube-api-access-j69sq\") pod \"cad80b61-7c72-479d-9fc4-5c2fe5441474\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") "
Jul 23 00:57:21.898864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.898715    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-podinfo\") pod \"cad80b61-7c72-479d-9fc4-5c2fe5441474\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") "
Jul 23 00:57:21.898864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.898787    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-credential-socket\") pod \"cad80b61-7c72-479d-9fc4-5c2fe5441474\" (UID: \"cad80b61-7c72-479d-9fc4-5c2fe5441474\") "
Jul 23 00:57:21.898864 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.898838    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cad80b61-7c72-479d-9fc4-5c2fe5441474" (UID: "cad80b61-7c72-479d-9fc4-5c2fe5441474"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:57:21.899140 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.898917    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cad80b61-7c72-479d-9fc4-5c2fe5441474-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cad80b61-7c72-479d-9fc4-5c2fe5441474" (UID: "cad80b61-7c72-479d-9fc4-5c2fe5441474"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:57:21.899211 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.899189    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cad80b61-7c72-479d-9fc4-5c2fe5441474" (UID: "cad80b61-7c72-479d-9fc4-5c2fe5441474"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:57:21.899266 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.899237    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:57:21.899266 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.899262    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cad80b61-7c72-479d-9fc4-5c2fe5441474-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:57:21.899413 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.899278    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:57:21.899723 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.899691    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-data" (OuterVolumeSpecName: "istio-data") pod "cad80b61-7c72-479d-9fc4-5c2fe5441474" (UID: "cad80b61-7c72-479d-9fc4-5c2fe5441474"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:57:21.901173 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.901148    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cad80b61-7c72-479d-9fc4-5c2fe5441474-kube-api-access-j69sq" (OuterVolumeSpecName: "kube-api-access-j69sq") pod "cad80b61-7c72-479d-9fc4-5c2fe5441474" (UID: "cad80b61-7c72-479d-9fc4-5c2fe5441474"). InnerVolumeSpecName "kube-api-access-j69sq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:57:21.901271 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.901152    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-token" (OuterVolumeSpecName: "istio-token") pod "cad80b61-7c72-479d-9fc4-5c2fe5441474" (UID: "cad80b61-7c72-479d-9fc4-5c2fe5441474"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:57:21.901474 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.901451    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cad80b61-7c72-479d-9fc4-5c2fe5441474" (UID: "cad80b61-7c72-479d-9fc4-5c2fe5441474"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:57:21.901546 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:21.901523    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cad80b61-7c72-479d-9fc4-5c2fe5441474" (UID: "cad80b61-7c72-479d-9fc4-5c2fe5441474"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 00:57:22.000215 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.000186    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j69sq\" (UniqueName: \"kubernetes.io/projected/cad80b61-7c72-479d-9fc4-5c2fe5441474-kube-api-access-j69sq\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:57:22.000215 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.000213    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:57:22.000215 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.000223    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:57:22.000483 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.000231    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:57:22.000483 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.000239    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:57:22.000483 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.000247    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cad80b61-7c72-479d-9fc4-5c2fe5441474-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:57:22.369779 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.369741    2561 generic.go:356] "Generic (PLEG): container finished" podID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerID="b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f" exitCode=0
Jul 23 00:57:22.369961 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.369795    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"
Jul 23 00:57:22.369961 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.369826    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" event={"ID":"cad80b61-7c72-479d-9fc4-5c2fe5441474","Type":"ContainerDied","Data":"b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f"}
Jul 23 00:57:22.369961 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.369883    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h" event={"ID":"cad80b61-7c72-479d-9fc4-5c2fe5441474","Type":"ContainerDied","Data":"dbcc66077aefb1095310ab81b356c7138d09acb6f24244db977d92f838d42db0"}
Jul 23 00:57:22.369961 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.369905    2561 scope.go:117] "RemoveContainer" containerID="b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f"
Jul 23 00:57:22.383147 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.383119    2561 scope.go:117] "RemoveContainer" containerID="fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6"
Jul 23 00:57:22.391071 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.391050    2561 scope.go:117] "RemoveContainer" containerID="b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f"
Jul 23 00:57:22.391381 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:57:22.391339    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f\": container with ID starting with b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f not found: ID does not exist" containerID="b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f"
Jul 23 00:57:22.391452 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.391384    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f"} err="failed to get container status \"b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f\": rpc error: code = NotFound desc = could not find container \"b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f\": container with ID starting with b015d27131a7fa2880d965c8ffca74eadc7493d19a7d713d058c9f5f3296e43f not found: ID does not exist"
Jul 23 00:57:22.391452 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.391407    2561 scope.go:117] "RemoveContainer" containerID="fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6"
Jul 23 00:57:22.391676 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:57:22.391655    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6\": container with ID starting with fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6 not found: ID does not exist" containerID="fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6"
Jul 23 00:57:22.391725 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.391683    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6"} err="failed to get container status \"fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6\": rpc error: code = NotFound desc = could not find container \"fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6\": container with ID starting with fa45b5408f0ac79d7b523934f0a44a3932f022fbafbc8e6cdf37618cc9f67df6 not found: ID does not exist"
Jul 23 00:57:22.402215 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.402186    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"]
Jul 23 00:57:22.406547 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.406525    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-882c377a-openshift-default-7d84cbf457-8dm7h"]
Jul 23 00:57:22.786996 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:22.786960    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" path="/var/lib/kubelet/pods/cad80b61-7c72-479d-9fc4-5c2fe5441474/volumes"
Jul 23 00:57:34.354294 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:34.354259    2561 patch_prober.go:28] interesting pod/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 23 00:57:34.354743 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:34.354319    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" podUID="074ef508-6034-4756-8f68-a234740cc2ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 23 00:57:34.431126 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:34.431090    2561 generic.go:356] "Generic (PLEG): container finished" podID="074ef508-6034-4756-8f68-a234740cc2ae" containerID="4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979" exitCode=0
Jul 23 00:57:34.431295 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:34.431168    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" event={"ID":"074ef508-6034-4756-8f68-a234740cc2ae","Type":"ContainerDied","Data":"4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979"}
Jul 23 00:57:34.431675 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:34.431660    2561 scope.go:117] "RemoveContainer" containerID="4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979"
Jul 23 00:57:34.432829 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:34.432800    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:57:34.432900 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:34.432885    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:57:34.432939 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:34.432929    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:57:35.448412 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:35.444470    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" event={"ID":"074ef508-6034-4756-8f68-a234740cc2ae","Type":"ContainerStarted","Data":"2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f"}
Jul 23 00:57:35.672021 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:35.671982    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:35.672243 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:35.672215    2561 patch_prober.go:28] interesting pod/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 23 00:57:35.672321 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:35.672281    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" podUID="074ef508-6034-4756-8f68-a234740cc2ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 23 00:57:36.671557 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:36.671516    2561 patch_prober.go:28] interesting pod/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 23 00:57:36.671975 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:36.671574    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" podUID="074ef508-6034-4756-8f68-a234740cc2ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 23 00:57:37.671397 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:37.671281    2561 patch_prober.go:28] interesting pod/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 23 00:57:37.671564 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:37.671357    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" podUID="074ef508-6034-4756-8f68-a234740cc2ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 23 00:57:38.672049 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:38.672009    2561 patch_prober.go:28] interesting pod/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 23 00:57:38.672511 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:38.672068    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" podUID="074ef508-6034-4756-8f68-a234740cc2ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 23 00:57:39.675648 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:39.675619    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:39.676033 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:39.675867    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:39.676628 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:39.676610    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:57:44.588449 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.588409    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k"]
Jul 23 00:57:44.588836 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.588762    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerName="istio-proxy"
Jul 23 00:57:44.588836 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.588772    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerName="istio-proxy"
Jul 23 00:57:44.588836 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.588782    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerName="istio-proxy"
Jul 23 00:57:44.588836 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.588789    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerName="istio-proxy"
Jul 23 00:57:44.588969 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.588854    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerName="istio-proxy"
Jul 23 00:57:44.590871 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.590855    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k"
Jul 23 00:57:44.593325 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.593306    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4zd6c\""
Jul 23 00:57:44.599358 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.599320    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k"]
Jul 23 00:57:44.711231 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.711199    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee7875b3-3356-463a-aee9-18756e860c54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k\" (UID: \"ee7875b3-3356-463a-aee9-18756e860c54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k"
Jul 23 00:57:44.812330 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.812299    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee7875b3-3356-463a-aee9-18756e860c54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k\" (UID: \"ee7875b3-3356-463a-aee9-18756e860c54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k"
Jul 23 00:57:44.812885 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.812864    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee7875b3-3356-463a-aee9-18756e860c54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k\" (UID: \"ee7875b3-3356-463a-aee9-18756e860c54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k"
Jul 23 00:57:44.901885 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:44.901787    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k"
Jul 23 00:57:45.037803 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:45.037778    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k"]
Jul 23 00:57:45.040187 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:57:45.040156    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7875b3_3356_463a_aee9_18756e860c54.slice/crio-bf2ff8b7a03bcd9b444c90cabe9528ae03d0284d7c289c798a6af275d30bd38b WatchSource:0}: Error finding container bf2ff8b7a03bcd9b444c90cabe9528ae03d0284d7c289c798a6af275d30bd38b: Status 404 returned error can't find the container with id bf2ff8b7a03bcd9b444c90cabe9528ae03d0284d7c289c798a6af275d30bd38b
Jul 23 00:57:45.495960 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:45.495867    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k" event={"ID":"ee7875b3-3356-463a-aee9-18756e860c54","Type":"ContainerStarted","Data":"b2b433ab0ef98f0b885571e014dc803671ef57f393ac3e5b5cc4b975601e179c"}
Jul 23 00:57:45.495960 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:45.495928    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k" event={"ID":"ee7875b3-3356-463a-aee9-18756e860c54","Type":"ContainerStarted","Data":"bf2ff8b7a03bcd9b444c90cabe9528ae03d0284d7c289c798a6af275d30bd38b"}
Jul 23 00:57:45.512568 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:45.512524    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k" podStartSLOduration=1.512508392 podStartE2EDuration="1.512508392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:57:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:57:45.511177869 +0000 UTC m=+2069.317589102" watchObservedRunningTime="2026-07-23 00:57:45.512508392 +0000 UTC m=+2069.318919627"
Jul 23 00:57:46.501479 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:46.501446    2561 generic.go:356] "Generic (PLEG): container finished" podID="ee7875b3-3356-463a-aee9-18756e860c54" containerID="b2b433ab0ef98f0b885571e014dc803671ef57f393ac3e5b5cc4b975601e179c" exitCode=0
Jul 23 00:57:46.501875 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:46.501529    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k" event={"ID":"ee7875b3-3356-463a-aee9-18756e860c54","Type":"ContainerDied","Data":"b2b433ab0ef98f0b885571e014dc803671ef57f393ac3e5b5cc4b975601e179c"}
Jul 23 00:57:47.639255 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:47.639229    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k"
Jul 23 00:57:47.737997 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:47.737963    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee7875b3-3356-463a-aee9-18756e860c54-maas-api-service-ca\") pod \"ee7875b3-3356-463a-aee9-18756e860c54\" (UID: \"ee7875b3-3356-463a-aee9-18756e860c54\") "
Jul 23 00:57:47.738400 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:47.738343    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee7875b3-3356-463a-aee9-18756e860c54-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ee7875b3-3356-463a-aee9-18756e860c54" (UID: "ee7875b3-3356-463a-aee9-18756e860c54"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:57:47.839602 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:47.839517    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee7875b3-3356-463a-aee9-18756e860c54-maas-api-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:57:48.512099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:48.512069    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k"
Jul 23 00:57:48.512099 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:48.512085    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k" event={"ID":"ee7875b3-3356-463a-aee9-18756e860c54","Type":"ContainerDied","Data":"bf2ff8b7a03bcd9b444c90cabe9528ae03d0284d7c289c798a6af275d30bd38b"}
Jul 23 00:57:48.512314 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:48.512128    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf2ff8b7a03bcd9b444c90cabe9528ae03d0284d7c289c798a6af275d30bd38b"
Jul 23 00:57:59.670760 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:59.670720    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k"]
Jul 23 00:57:59.673332 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:59.673305    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cc4a0bee-qv62k"]
Jul 23 00:57:59.812931 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:59.812891    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"]
Jul 23 00:57:59.813123 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:57:59.813096    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" podUID="074ef508-6034-4756-8f68-a234740cc2ae" containerName="istio-proxy" containerID="cri-o://2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f" gracePeriod=30
Jul 23 00:58:00.787609 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:00.787575    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee7875b3-3356-463a-aee9-18756e860c54" path="/var/lib/kubelet/pods/ee7875b3-3356-463a-aee9-18756e860c54/volumes"
Jul 23 00:58:05.068246 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.068221    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:58:05.172654 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.172566    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/074ef508-6034-4756-8f68-a234740cc2ae-istio-podinfo\") pod \"074ef508-6034-4756-8f68-a234740cc2ae\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") "
Jul 23 00:58:05.172654 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.172608    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-credential-socket\") pod \"074ef508-6034-4756-8f68-a234740cc2ae\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") "
Jul 23 00:58:05.172654 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.172653    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-workload-certs\") pod \"074ef508-6034-4756-8f68-a234740cc2ae\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") "
Jul 23 00:58:05.172941 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.172688    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/074ef508-6034-4756-8f68-a234740cc2ae-istio-token\") pod \"074ef508-6034-4756-8f68-a234740cc2ae\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") "
Jul 23 00:58:05.172941 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.172715    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pm9zr\" (UniqueName: \"kubernetes.io/projected/074ef508-6034-4756-8f68-a234740cc2ae-kube-api-access-pm9zr\") pod \"074ef508-6034-4756-8f68-a234740cc2ae\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") "
Jul 23 00:58:05.172941 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.172738    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-istio-data\") pod \"074ef508-6034-4756-8f68-a234740cc2ae\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") "
Jul 23 00:58:05.172941 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.172766    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-istio-envoy\") pod \"074ef508-6034-4756-8f68-a234740cc2ae\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") "
Jul 23 00:58:05.172941 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.172800    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/074ef508-6034-4756-8f68-a234740cc2ae-istiod-ca-cert\") pod \"074ef508-6034-4756-8f68-a234740cc2ae\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") "
Jul 23 00:58:05.172941 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.172845    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-workload-socket\") pod \"074ef508-6034-4756-8f68-a234740cc2ae\" (UID: \"074ef508-6034-4756-8f68-a234740cc2ae\") "
Jul 23 00:58:05.173232 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.173002    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "074ef508-6034-4756-8f68-a234740cc2ae" (UID: "074ef508-6034-4756-8f68-a234740cc2ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:58:05.173232 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.173146    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "074ef508-6034-4756-8f68-a234740cc2ae" (UID: "074ef508-6034-4756-8f68-a234740cc2ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:58:05.173232 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.173165    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:58:05.173423 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.173269    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "074ef508-6034-4756-8f68-a234740cc2ae" (UID: "074ef508-6034-4756-8f68-a234740cc2ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:58:05.173534 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.173511    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/074ef508-6034-4756-8f68-a234740cc2ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "074ef508-6034-4756-8f68-a234740cc2ae" (UID: "074ef508-6034-4756-8f68-a234740cc2ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:58:05.173993 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.173964    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "074ef508-6034-4756-8f68-a234740cc2ae" (UID: "074ef508-6034-4756-8f68-a234740cc2ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:58:05.175081 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.175057    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/074ef508-6034-4756-8f68-a234740cc2ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "074ef508-6034-4756-8f68-a234740cc2ae" (UID: "074ef508-6034-4756-8f68-a234740cc2ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:58:05.175308 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.175286    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/074ef508-6034-4756-8f68-a234740cc2ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "074ef508-6034-4756-8f68-a234740cc2ae" (UID: "074ef508-6034-4756-8f68-a234740cc2ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 00:58:05.175405 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.175323    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/074ef508-6034-4756-8f68-a234740cc2ae-kube-api-access-pm9zr" (OuterVolumeSpecName: "kube-api-access-pm9zr") pod "074ef508-6034-4756-8f68-a234740cc2ae" (UID: "074ef508-6034-4756-8f68-a234740cc2ae"). InnerVolumeSpecName "kube-api-access-pm9zr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:58:05.175454 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.175407    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "074ef508-6034-4756-8f68-a234740cc2ae" (UID: "074ef508-6034-4756-8f68-a234740cc2ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:58:05.274311 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.274281    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/074ef508-6034-4756-8f68-a234740cc2ae-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:58:05.274311 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.274306    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:58:05.274311 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.274317    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/074ef508-6034-4756-8f68-a234740cc2ae-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:58:05.274541 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.274327    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm9zr\" (UniqueName: \"kubernetes.io/projected/074ef508-6034-4756-8f68-a234740cc2ae-kube-api-access-pm9zr\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:58:05.274541 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.274338    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:58:05.274541 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.274351    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:58:05.274541 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.274379    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/074ef508-6034-4756-8f68-a234740cc2ae-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:58:05.274541 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.274387    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/074ef508-6034-4756-8f68-a234740cc2ae-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 00:58:05.594810 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.594775    2561 generic.go:356] "Generic (PLEG): container finished" podID="074ef508-6034-4756-8f68-a234740cc2ae" containerID="2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f" exitCode=0
Jul 23 00:58:05.595015 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.594824    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"
Jul 23 00:58:05.595015 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.594868    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" event={"ID":"074ef508-6034-4756-8f68-a234740cc2ae","Type":"ContainerDied","Data":"2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f"}
Jul 23 00:58:05.595015 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.594937    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7" event={"ID":"074ef508-6034-4756-8f68-a234740cc2ae","Type":"ContainerDied","Data":"b7893381a5f7a06c003811123d7b225f6c3778cbdf71829758ec6b9425c6512e"}
Jul 23 00:58:05.595015 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.594968    2561 scope.go:117] "RemoveContainer" containerID="2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f"
Jul 23 00:58:05.608620 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.608593    2561 scope.go:117] "RemoveContainer" containerID="4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979"
Jul 23 00:58:05.617868 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.617847    2561 scope.go:117] "RemoveContainer" containerID="2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f"
Jul 23 00:58:05.618132 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:58:05.618111    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f\": container with ID starting with 2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f not found: ID does not exist" containerID="2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f"
Jul 23 00:58:05.618228 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.618143    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f"} err="failed to get container status \"2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f\": rpc error: code = NotFound desc = could not find container \"2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f\": container with ID starting with 2625cef2dee61e61225690e474a818f64a043a17037a0b06cdf48f193558801f not found: ID does not exist"
Jul 23 00:58:05.618228 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.618165    2561 scope.go:117] "RemoveContainer" containerID="4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979"
Jul 23 00:58:05.618435 ip-10-0-133-215 kubenswrapper[2561]: E0723 00:58:05.618419    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979\": container with ID starting with 4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979 not found: ID does not exist" containerID="4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979"
Jul 23 00:58:05.618484 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.618441    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979"} err="failed to get container status \"4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979\": rpc error: code = NotFound desc = could not find container \"4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979\": container with ID starting with 4ae337d6ee770520ef42d0ee1e831f8decd3d52381cb77e7ffa6d1d54b0d0979 not found: ID does not exist"
Jul 23 00:58:05.623313 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.623293    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"]
Jul 23 00:58:05.627787 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:05.627765    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-cc4a0bee-openshift-default-79cb68948c-pzsf7"]
Jul 23 00:58:06.787654 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:06.787619    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="074ef508-6034-4756-8f68-a234740cc2ae" path="/var/lib/kubelet/pods/074ef508-6034-4756-8f68-a234740cc2ae/volumes"
Jul 23 00:58:16.805895 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:16.805865    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:58:16.808187 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:58:16.808168    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 00:59:37.950190 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.950115    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"]
Jul 23 00:59:37.950671 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.950490    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee7875b3-3356-463a-aee9-18756e860c54" containerName="revoke-keys"
Jul 23 00:59:37.950671 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.950502    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee7875b3-3356-463a-aee9-18756e860c54" containerName="revoke-keys"
Jul 23 00:59:37.950671 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.950512    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="074ef508-6034-4756-8f68-a234740cc2ae" containerName="istio-proxy"
Jul 23 00:59:37.950671 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.950517    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="074ef508-6034-4756-8f68-a234740cc2ae" containerName="istio-proxy"
Jul 23 00:59:37.950671 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.950522    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="074ef508-6034-4756-8f68-a234740cc2ae" containerName="istio-proxy"
Jul 23 00:59:37.950671 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.950527    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="074ef508-6034-4756-8f68-a234740cc2ae" containerName="istio-proxy"
Jul 23 00:59:37.950671 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.950586    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cad80b61-7c72-479d-9fc4-5c2fe5441474" containerName="istio-proxy"
Jul 23 00:59:37.950671 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.950594    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="074ef508-6034-4756-8f68-a234740cc2ae" containerName="istio-proxy"
Jul 23 00:59:37.950671 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.950602    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee7875b3-3356-463a-aee9-18756e860c54" containerName="revoke-keys"
Jul 23 00:59:37.952544 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.952526    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:37.955814 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.955791    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-489ccc2d-openshift-default-dockercfg-42w5r\""
Jul 23 00:59:37.964282 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.964258    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-data\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:37.964378 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.964292    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-podinfo\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:37.964378 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.964326    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-workload-socket\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:37.964378 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.964350    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8xq7\" (UniqueName: \"kubernetes.io/projected/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-kube-api-access-p8xq7\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:37.964492 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.964393    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-credential-socket\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:37.964492 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.964431    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-workload-certs\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:37.964492 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.964466    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-token\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:37.964589 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.964556    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-envoy\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:37.964589 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.964584    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istiod-ca-cert\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:37.973287 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:37.973264    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"]
Jul 23 00:59:38.065452 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.065418    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-workload-socket\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.065452 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.065453    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8xq7\" (UniqueName: \"kubernetes.io/projected/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-kube-api-access-p8xq7\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.065713 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.065469    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-credential-socket\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.065713 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.065504    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-workload-certs\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.065713 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.065537    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-token\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.065713 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.065562    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-envoy\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.065713 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.065579    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istiod-ca-cert\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.065713 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.065621    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-data\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.065713 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.065648    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-podinfo\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.066084 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.065857    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-workload-socket\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.066084 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.065902    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-workload-certs\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.066084 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.065933    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-credential-socket\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.066222 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.066107    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-data\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.066320 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.066259    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istiod-ca-cert\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.067965 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.067945    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-envoy\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.068124 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.068108    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-podinfo\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.073567 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.073540    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8xq7\" (UniqueName: \"kubernetes.io/projected/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-kube-api-access-p8xq7\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.073762 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.073703    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-token\") pod \"e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") " pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.262586 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.262504    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:38.402826 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.402797    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"]
Jul 23 00:59:38.404547 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:59:38.404516    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e96a5e_a2ed_47a4_9dc3_8c238f3e0ba2.slice/crio-a377d63e043e96a307d3f82fdc17046567a4dcab08b342d6f010ca2f581c92ee WatchSource:0}: Error finding container a377d63e043e96a307d3f82fdc17046567a4dcab08b342d6f010ca2f581c92ee: Status 404 returned error can't find the container with id a377d63e043e96a307d3f82fdc17046567a4dcab08b342d6f010ca2f581c92ee
Jul 23 00:59:38.406856 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.406820    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:59:38.406961 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.406889    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:59:38.406961 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:38.406918    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:59:39.037127 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:39.037095    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" event={"ID":"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2","Type":"ContainerStarted","Data":"7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a"}
Jul 23 00:59:39.037566 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:39.037150    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" event={"ID":"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2","Type":"ContainerStarted","Data":"a377d63e043e96a307d3f82fdc17046567a4dcab08b342d6f010ca2f581c92ee"}
Jul 23 00:59:39.060792 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:39.060738    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" podStartSLOduration=2.060718974 podStartE2EDuration="2.060718974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:59:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:59:39.0588844 +0000 UTC m=+2182.865295673" watchObservedRunningTime="2026-07-23 00:59:39.060718974 +0000 UTC m=+2182.867130242"
Jul 23 00:59:39.263102 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:39.263064    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:39.268325 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:39.268301    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:40.042690 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:40.042656    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:40.043574 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:40.043554    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:41.232212 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.232180    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"]
Jul 23 00:59:41.232655 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.232639    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="074ef508-6034-4756-8f68-a234740cc2ae" containerName="istio-proxy"
Jul 23 00:59:41.234493 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.234478    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:41.237527 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.237493    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-489ccc2d-serving-cert\""
Jul 23 00:59:41.249035 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.249006    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"]
Jul 23 00:59:41.292029 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.291997    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/724a9634-81ce-4cdc-9298-696ea05039f0-maas-api-tls\") pod \"maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd\" (UID: \"724a9634-81ce-4cdc-9298-696ea05039f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:41.292242 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.292081    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/724a9634-81ce-4cdc-9298-696ea05039f0-openshift-service-ca\") pod \"maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd\" (UID: \"724a9634-81ce-4cdc-9298-696ea05039f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:41.292242 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.292154    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8d2g\" (UniqueName: \"kubernetes.io/projected/724a9634-81ce-4cdc-9298-696ea05039f0-kube-api-access-t8d2g\") pod \"maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd\" (UID: \"724a9634-81ce-4cdc-9298-696ea05039f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:41.392844 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.392798    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/724a9634-81ce-4cdc-9298-696ea05039f0-maas-api-tls\") pod \"maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd\" (UID: \"724a9634-81ce-4cdc-9298-696ea05039f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:41.393045 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.392889    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/724a9634-81ce-4cdc-9298-696ea05039f0-openshift-service-ca\") pod \"maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd\" (UID: \"724a9634-81ce-4cdc-9298-696ea05039f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:41.393045 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.392964    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8d2g\" (UniqueName: \"kubernetes.io/projected/724a9634-81ce-4cdc-9298-696ea05039f0-kube-api-access-t8d2g\") pod \"maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd\" (UID: \"724a9634-81ce-4cdc-9298-696ea05039f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:41.393609 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.393585    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/724a9634-81ce-4cdc-9298-696ea05039f0-openshift-service-ca\") pod \"maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd\" (UID: \"724a9634-81ce-4cdc-9298-696ea05039f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:41.395503 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.395480    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/724a9634-81ce-4cdc-9298-696ea05039f0-maas-api-tls\") pod \"maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd\" (UID: \"724a9634-81ce-4cdc-9298-696ea05039f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:41.403334 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.403309    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8d2g\" (UniqueName: \"kubernetes.io/projected/724a9634-81ce-4cdc-9298-696ea05039f0-kube-api-access-t8d2g\") pod \"maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd\" (UID: \"724a9634-81ce-4cdc-9298-696ea05039f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:41.545460 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.545358    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:41.891494 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:41.891416    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"]
Jul 23 00:59:41.894750 ip-10-0-133-215 kubenswrapper[2561]: W0723 00:59:41.894717    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod724a9634_81ce_4cdc_9298_696ea05039f0.slice/crio-bb1a0260943e7cca643bd4184321469441e3d5cfcb75898b04923dbe443cf255 WatchSource:0}: Error finding container bb1a0260943e7cca643bd4184321469441e3d5cfcb75898b04923dbe443cf255: Status 404 returned error can't find the container with id bb1a0260943e7cca643bd4184321469441e3d5cfcb75898b04923dbe443cf255
Jul 23 00:59:42.052430 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:42.052391    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd" event={"ID":"724a9634-81ce-4cdc-9298-696ea05039f0","Type":"ContainerStarted","Data":"bb1a0260943e7cca643bd4184321469441e3d5cfcb75898b04923dbe443cf255"}
Jul 23 00:59:43.059335 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:43.059300    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd" event={"ID":"724a9634-81ce-4cdc-9298-696ea05039f0","Type":"ContainerStarted","Data":"4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d"}
Jul 23 00:59:43.059786 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:43.059402    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:43.077256 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:43.077206    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd" podStartSLOduration=1.7101721539999999 podStartE2EDuration="2.077189272s" totalImagesPullingTime="367.017118ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:59:43.07542191 +0000 UTC m=+2186.881833140" watchObservedRunningTime="2026-07-23 00:59:43.077189272 +0000 UTC m=+2186.883600504"
Jul 23 00:59:49.069883 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:49.069855    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 00:59:54.112324 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:54.112287    2561 generic.go:356] "Generic (PLEG): container finished" podID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerID="7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a" exitCode=0
Jul 23 00:59:54.112324 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:54.112329    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" event={"ID":"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2","Type":"ContainerDied","Data":"7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a"}
Jul 23 00:59:54.112948 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:54.112876    2561 scope.go:117] "RemoveContainer" containerID="7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a"
Jul 23 00:59:54.113924 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:54.113893    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:59:54.114021 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:54.113979    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:59:54.114084 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:54.114043    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:59:55.043020 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:55.042917    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:55.118730 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:55.118692    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" event={"ID":"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2","Type":"ContainerStarted","Data":"29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0"}
Jul 23 00:59:55.263608 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:55.263567    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:55.264860 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:55.264829    2561 patch_prober.go:28] interesting pod/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 23 00:59:55.265004 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:55.264886    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 23 00:59:56.263426 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:56.263350    2561 patch_prober.go:28] interesting pod/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 23 00:59:56.263822 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:56.263456    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 23 00:59:57.263810 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:57.263775    2561 patch_prober.go:28] interesting pod/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 23 00:59:57.264200 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:57.263844    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 23 00:59:58.263631 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:58.263587    2561 patch_prober.go:28] interesting pod/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 23 00:59:58.263824 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:58.263667    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 23 00:59:59.266924 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:59.266894    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:59.267322 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:59.267182    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 00:59:59.267867 ip-10-0-133-215 kubenswrapper[2561]: I0723 00:59:59.267852    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 01:00:02.629036 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:02.628976    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"]
Jul 23 01:00:02.629964 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:02.629901    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd" podUID="724a9634-81ce-4cdc-9298-696ea05039f0" containerName="maas-api" containerID="cri-o://4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d" gracePeriod=30
Jul 23 01:00:02.888798 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:02.888744    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 01:00:02.983133 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:02.983093    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8d2g\" (UniqueName: \"kubernetes.io/projected/724a9634-81ce-4cdc-9298-696ea05039f0-kube-api-access-t8d2g\") pod \"724a9634-81ce-4cdc-9298-696ea05039f0\" (UID: \"724a9634-81ce-4cdc-9298-696ea05039f0\") "
Jul 23 01:00:02.983325 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:02.983155    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/724a9634-81ce-4cdc-9298-696ea05039f0-openshift-service-ca\") pod \"724a9634-81ce-4cdc-9298-696ea05039f0\" (UID: \"724a9634-81ce-4cdc-9298-696ea05039f0\") "
Jul 23 01:00:02.983325 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:02.983242    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/724a9634-81ce-4cdc-9298-696ea05039f0-maas-api-tls\") pod \"724a9634-81ce-4cdc-9298-696ea05039f0\" (UID: \"724a9634-81ce-4cdc-9298-696ea05039f0\") "
Jul 23 01:00:02.983598 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:02.983570    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/724a9634-81ce-4cdc-9298-696ea05039f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "724a9634-81ce-4cdc-9298-696ea05039f0" (UID: "724a9634-81ce-4cdc-9298-696ea05039f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:00:02.985242 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:02.985216    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/724a9634-81ce-4cdc-9298-696ea05039f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "724a9634-81ce-4cdc-9298-696ea05039f0" (UID: "724a9634-81ce-4cdc-9298-696ea05039f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 01:00:02.985375 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:02.985300    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/724a9634-81ce-4cdc-9298-696ea05039f0-kube-api-access-t8d2g" (OuterVolumeSpecName: "kube-api-access-t8d2g") pod "724a9634-81ce-4cdc-9298-696ea05039f0" (UID: "724a9634-81ce-4cdc-9298-696ea05039f0"). InnerVolumeSpecName "kube-api-access-t8d2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:00:03.084840 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:03.084805    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/724a9634-81ce-4cdc-9298-696ea05039f0-maas-api-tls\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:00:03.084840 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:03.084832    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8d2g\" (UniqueName: \"kubernetes.io/projected/724a9634-81ce-4cdc-9298-696ea05039f0-kube-api-access-t8d2g\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:00:03.084840 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:03.084844    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/724a9634-81ce-4cdc-9298-696ea05039f0-openshift-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:00:03.162294 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:03.162206    2561 generic.go:356] "Generic (PLEG): container finished" podID="724a9634-81ce-4cdc-9298-696ea05039f0" containerID="4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d" exitCode=0
Jul 23 01:00:03.162294 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:03.162246    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"
Jul 23 01:00:03.162523 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:03.162299    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd" event={"ID":"724a9634-81ce-4cdc-9298-696ea05039f0","Type":"ContainerDied","Data":"4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d"}
Jul 23 01:00:03.162523 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:03.162346    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd" event={"ID":"724a9634-81ce-4cdc-9298-696ea05039f0","Type":"ContainerDied","Data":"bb1a0260943e7cca643bd4184321469441e3d5cfcb75898b04923dbe443cf255"}
Jul 23 01:00:03.162523 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:03.162397    2561 scope.go:117] "RemoveContainer" containerID="4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d"
Jul 23 01:00:03.171114 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:03.171097    2561 scope.go:117] "RemoveContainer" containerID="4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d"
Jul 23 01:00:03.171393 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:00:03.171372    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d\": container with ID starting with 4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d not found: ID does not exist" containerID="4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d"
Jul 23 01:00:03.171444 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:03.171402    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d"} err="failed to get container status \"4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d\": rpc error: code = NotFound desc = could not find container \"4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d\": container with ID starting with 4358cb10dbef169b321e69f86ead9f02bd86e11f5092a6b49a9177eed1ef789d not found: ID does not exist"
Jul 23 01:00:03.186679 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:03.186650    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"]
Jul 23 01:00:03.191225 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:03.191203    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-489ccc2d-57cdf64f85-5kbzd"]
Jul 23 01:00:04.787332 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:04.787289    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="724a9634-81ce-4cdc-9298-696ea05039f0" path="/var/lib/kubelet/pods/724a9634-81ce-4cdc-9298-696ea05039f0/volumes"
Jul 23 01:00:13.995950 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:13.995905    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"]
Jul 23 01:00:13.996533 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:13.996226    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerName="istio-proxy" containerID="cri-o://29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0" gracePeriod=30
Jul 23 01:00:14.269484 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:14.269389    2561 patch_prober.go:28] interesting pod/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 23 01:00:14.269484 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:14.269457    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 01:00:19.240070 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.240041    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 01:00:19.242142 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.242116    2561 generic.go:356] "Generic (PLEG): container finished" podID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerID="29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0" exitCode=0
Jul 23 01:00:19.242249 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.242155    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"
Jul 23 01:00:19.242249 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.242175    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" event={"ID":"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2","Type":"ContainerDied","Data":"29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0"}
Jul 23 01:00:19.242249 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.242230    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n" event={"ID":"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2","Type":"ContainerDied","Data":"a377d63e043e96a307d3f82fdc17046567a4dcab08b342d6f010ca2f581c92ee"}
Jul 23 01:00:19.242356 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.242254    2561 scope.go:117] "RemoveContainer" containerID="29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0"
Jul 23 01:00:19.250303 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.250285    2561 scope.go:117] "RemoveContainer" containerID="7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a"
Jul 23 01:00:19.257763 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.257745    2561 scope.go:117] "RemoveContainer" containerID="29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0"
Jul 23 01:00:19.258028 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:00:19.258007    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0\": container with ID starting with 29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0 not found: ID does not exist" containerID="29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0"
Jul 23 01:00:19.258101 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.258042    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0"} err="failed to get container status \"29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0\": rpc error: code = NotFound desc = could not find container \"29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0\": container with ID starting with 29b1a5dc8b89e21963194e00627256dd1d87fa1924a18314644e98b1ac932de0 not found: ID does not exist"
Jul 23 01:00:19.258101 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.258067    2561 scope.go:117] "RemoveContainer" containerID="7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a"
Jul 23 01:00:19.258328 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:00:19.258311    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a\": container with ID starting with 7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a not found: ID does not exist" containerID="7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a"
Jul 23 01:00:19.258386 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.258337    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a"} err="failed to get container status \"7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a\": rpc error: code = NotFound desc = could not find container \"7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a\": container with ID starting with 7f1dfdf8efc43037e646de11caaf2bbc7802920a250476a3fa34b173bd74d15a not found: ID does not exist"
Jul 23 01:00:19.326957 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.326866    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-token\") pod \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") "
Jul 23 01:00:19.326957 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.326908    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istiod-ca-cert\") pod \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") "
Jul 23 01:00:19.326957 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.326939    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-data\") pod \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") "
Jul 23 01:00:19.327238 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.326967    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8xq7\" (UniqueName: \"kubernetes.io/projected/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-kube-api-access-p8xq7\") pod \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") "
Jul 23 01:00:19.327238 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327000    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-credential-socket\") pod \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") "
Jul 23 01:00:19.327238 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327054    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-workload-socket\") pod \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") "
Jul 23 01:00:19.327238 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327102    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-workload-certs\") pod \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") "
Jul 23 01:00:19.327238 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327129    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-envoy\") pod \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") "
Jul 23 01:00:19.327238 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327165    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-podinfo\") pod \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\" (UID: \"71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2\") "
Jul 23 01:00:19.327580 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327388    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" (UID: "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:00:19.327580 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327405    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" (UID: "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:00:19.327580 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327418    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" (UID: "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:00:19.327580 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327426    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" (UID: "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:00:19.327580 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327543    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:00:19.327580 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327562    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:00:19.327580 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327576    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:00:19.327944 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327589    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:00:19.327944 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.327782    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-data" (OuterVolumeSpecName: "istio-data") pod "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" (UID: "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:00:19.329458 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.329426    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" (UID: "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:00:19.329671 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.329645    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" (UID: "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 01:00:19.329671 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.329659    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-kube-api-access-p8xq7" (OuterVolumeSpecName: "kube-api-access-p8xq7") pod "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" (UID: "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2"). InnerVolumeSpecName "kube-api-access-p8xq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:00:19.329816 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.329669    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-token" (OuterVolumeSpecName: "istio-token") pod "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" (UID: "71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:00:19.428725 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.428686    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:00:19.428725 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.428718    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:00:19.428725 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.428728    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:00:19.428725 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.428739    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:00:19.429001 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.428747    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8xq7\" (UniqueName: \"kubernetes.io/projected/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2-kube-api-access-p8xq7\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:00:19.564027 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.563997    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"]
Jul 23 01:00:19.568037 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:19.568013    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-489ccc2d-openshift-default-fdc66cc79-vhw8n"]
Jul 23 01:00:20.786912 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:20.786877    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" path="/var/lib/kubelet/pods/71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2/volumes"
Jul 23 01:00:33.461138 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.461099    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"]
Jul 23 01:00:33.461570 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.461436    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerName="istio-proxy"
Jul 23 01:00:33.461570 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.461448    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerName="istio-proxy"
Jul 23 01:00:33.461570 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.461467    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="724a9634-81ce-4cdc-9298-696ea05039f0" containerName="maas-api"
Jul 23 01:00:33.461570 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.461472    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="724a9634-81ce-4cdc-9298-696ea05039f0" containerName="maas-api"
Jul 23 01:00:33.461570 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.461523    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="724a9634-81ce-4cdc-9298-696ea05039f0" containerName="maas-api"
Jul 23 01:00:33.461570 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.461532    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerName="istio-proxy"
Jul 23 01:00:33.464596 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.464577    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.467347 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.467328    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0828012d-openshift-default-dockercfg-lsb52\""
Jul 23 01:00:33.478547 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.478523    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"]
Jul 23 01:00:33.646964 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.646922    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-envoy\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.647142 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.646994    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-workload-certs\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.647142 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.647030    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jdc2\" (UniqueName: \"kubernetes.io/projected/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-kube-api-access-4jdc2\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.647142 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.647060    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-workload-socket\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.647142 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.647083    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istiod-ca-cert\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.647142 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.647105    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-token\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.647313 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.647162    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-data\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.647313 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.647185    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-podinfo\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.647313 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.647208    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-credential-socket\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748088 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748004    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-workload-socket\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748088 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748045    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istiod-ca-cert\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748088 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748074    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-token\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748353 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748123    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-data\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748353 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748147    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-podinfo\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748353 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748174    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-credential-socket\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748353 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748336    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-envoy\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748617 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748437    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-workload-certs\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748617 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748494    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jdc2\" (UniqueName: \"kubernetes.io/projected/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-kube-api-access-4jdc2\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748617 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748502    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-workload-socket\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748617 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748576    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-credential-socket\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748817 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748642    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-data\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748817 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748792    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-workload-certs\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.748891 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.748867    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istiod-ca-cert\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.750766 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.750743    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-envoy\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.750959 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.750917    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-podinfo\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.756285 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.756253    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-token\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.756483 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.756465    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jdc2\" (UniqueName: \"kubernetes.io/projected/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-kube-api-access-4jdc2\") pod \"e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") " pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.774232 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.774209    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:33.909653 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.909625    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"]
Jul 23 01:00:33.911397 ip-10-0-133-215 kubenswrapper[2561]: W0723 01:00:33.911356    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa3852c_7e45_4f12_aadd_a9399fc5ebc6.slice/crio-e9a073e2461a80ccc6de1b5edae388fd191aa2cc939a0d094d0b2636cf1046aa WatchSource:0}: Error finding container e9a073e2461a80ccc6de1b5edae388fd191aa2cc939a0d094d0b2636cf1046aa: Status 404 returned error can't find the container with id e9a073e2461a80ccc6de1b5edae388fd191aa2cc939a0d094d0b2636cf1046aa
Jul 23 01:00:33.913488 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.913458    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:00:33.913594 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.913518    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:00:33.913594 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:33.913545    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:00:34.321079 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:34.321038    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj" event={"ID":"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6","Type":"ContainerStarted","Data":"809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20"}
Jul 23 01:00:34.321292 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:34.321117    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj" event={"ID":"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6","Type":"ContainerStarted","Data":"e9a073e2461a80ccc6de1b5edae388fd191aa2cc939a0d094d0b2636cf1046aa"}
Jul 23 01:00:34.341133 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:34.341072    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj" podStartSLOduration=1.341057812 podStartE2EDuration="1.341057812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:00:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:00:34.339448698 +0000 UTC m=+2238.145859930" watchObservedRunningTime="2026-07-23 01:00:34.341057812 +0000 UTC m=+2238.147469045"
Jul 23 01:00:34.774533 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:34.774500    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:34.779403 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:34.779376    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:35.325333 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:35.325303    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:35.326287 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:35.326272    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:36.335391 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.335335    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"]
Jul 23 01:00:36.335938 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.335917    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerName="istio-proxy"
Jul 23 01:00:36.336021 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.335942    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerName="istio-proxy"
Jul 23 01:00:36.336021 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.336017    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e96a5e-a2ed-47a4-9dc3-8c238f3e0ba2" containerName="istio-proxy"
Jul 23 01:00:36.339097 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.339074    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:36.341534 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.341512    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0828012d-serving-cert\""
Jul 23 01:00:36.348034 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.347985    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"]
Jul 23 01:00:36.372061 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.372026    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87745493-cd49-4f98-b0ad-c890126a898f-openshift-service-ca\") pod \"maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:36.372208 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.372073    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87745493-cd49-4f98-b0ad-c890126a898f-maas-api-tls\") pod \"maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:36.372208 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.372147    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg7cv\" (UniqueName: \"kubernetes.io/projected/87745493-cd49-4f98-b0ad-c890126a898f-kube-api-access-hg7cv\") pod \"maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:36.473389 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.473330    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87745493-cd49-4f98-b0ad-c890126a898f-openshift-service-ca\") pod \"maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:36.473574 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.473402    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87745493-cd49-4f98-b0ad-c890126a898f-maas-api-tls\") pod \"maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:36.473574 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.473433    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg7cv\" (UniqueName: \"kubernetes.io/projected/87745493-cd49-4f98-b0ad-c890126a898f-kube-api-access-hg7cv\") pod \"maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:36.473574 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:00:36.473546    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-serving-cert: secret "maas-api-e2e-mt-0828012d-serving-cert" not found
Jul 23 01:00:36.473696 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:00:36.473622    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87745493-cd49-4f98-b0ad-c890126a898f-maas-api-tls podName:87745493-cd49-4f98-b0ad-c890126a898f nodeName:}" failed. No retries permitted until 2026-07-23 01:00:36.973600469 +0000 UTC m=+2240.780011881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87745493-cd49-4f98-b0ad-c890126a898f-maas-api-tls") pod "maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx" (UID: "87745493-cd49-4f98-b0ad-c890126a898f") : secret "maas-api-e2e-mt-0828012d-serving-cert" not found
Jul 23 01:00:36.474079 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.474060    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87745493-cd49-4f98-b0ad-c890126a898f-openshift-service-ca\") pod \"maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:36.482418 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.482395    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg7cv\" (UniqueName: \"kubernetes.io/projected/87745493-cd49-4f98-b0ad-c890126a898f-kube-api-access-hg7cv\") pod \"maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:36.977209 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.977159    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87745493-cd49-4f98-b0ad-c890126a898f-maas-api-tls\") pod \"maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:36.979665 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:36.979642    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87745493-cd49-4f98-b0ad-c890126a898f-maas-api-tls\") pod \"maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:37.251241 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:37.251140    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:37.394100 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:37.394074    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"]
Jul 23 01:00:37.396029 ip-10-0-133-215 kubenswrapper[2561]: W0723 01:00:37.395994    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87745493_cd49_4f98_b0ad_c890126a898f.slice/crio-3664fe82dad2765776adbf69e5642879d593a17c24f1030d88a613a3c2f13d63 WatchSource:0}: Error finding container 3664fe82dad2765776adbf69e5642879d593a17c24f1030d88a613a3c2f13d63: Status 404 returned error can't find the container with id 3664fe82dad2765776adbf69e5642879d593a17c24f1030d88a613a3c2f13d63
Jul 23 01:00:37.397587 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:37.397558    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 01:00:38.342685 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:38.342650    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx" event={"ID":"87745493-cd49-4f98-b0ad-c890126a898f","Type":"ContainerStarted","Data":"ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc"}
Jul 23 01:00:38.342898 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:38.342709    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx" event={"ID":"87745493-cd49-4f98-b0ad-c890126a898f","Type":"ContainerStarted","Data":"3664fe82dad2765776adbf69e5642879d593a17c24f1030d88a613a3c2f13d63"}
Jul 23 01:00:38.342898 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:38.342830    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:38.361782 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:38.361732    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx" podStartSLOduration=2.003720796 podStartE2EDuration="2.361715995s" totalImagesPullingTime="357.995199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:00:38.359601922 +0000 UTC m=+2242.166013155" watchObservedRunningTime="2026-07-23 01:00:38.361715995 +0000 UTC m=+2242.168127228"
Jul 23 01:00:44.356844 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:44.356813    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:00:49.395395 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:49.395293    2561 generic.go:356] "Generic (PLEG): container finished" podID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerID="809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20" exitCode=0
Jul 23 01:00:49.395806 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:49.395394    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj" event={"ID":"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6","Type":"ContainerDied","Data":"809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20"}
Jul 23 01:00:49.395851 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:49.395819    2561 scope.go:117] "RemoveContainer" containerID="809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20"
Jul 23 01:00:49.396804 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:49.396775    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:00:49.396868 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:49.396844    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:00:49.396964 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:49.396873    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:00:50.325547 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:50.325510    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:50.402410 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:50.402350    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj" event={"ID":"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6","Type":"ContainerStarted","Data":"05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2"}
Jul 23 01:00:50.775311 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:50.775272    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:50.776586 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:50.776558    2561 patch_prober.go:28] interesting pod/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 23 01:00:50.776703 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:50.776610    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj" podUID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 23 01:00:51.774633 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:51.774595    2561 patch_prober.go:28] interesting pod/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 23 01:00:51.775038 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:51.774670    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj" podUID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 23 01:00:52.775238 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:52.775187    2561 patch_prober.go:28] interesting pod/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 23 01:00:52.775662 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:52.775269    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj" podUID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 23 01:00:53.775627 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:53.775580    2561 patch_prober.go:28] interesting pod/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Jul 23 01:00:53.776036 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:53.775650    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj" podUID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Jul 23 01:00:54.779154 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:54.779120    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:54.779563 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:54.779395    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:00:54.780216 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:00:54.780200    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:01:34.644053 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:34.644001    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"]
Jul 23 01:01:34.645221 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:34.644390    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx" podUID="87745493-cd49-4f98-b0ad-c890126a898f" containerName="maas-api" containerID="cri-o://ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc" gracePeriod=30
Jul 23 01:01:34.891197 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:34.891171    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:01:35.005729 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.005690    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87745493-cd49-4f98-b0ad-c890126a898f-maas-api-tls\") pod \"87745493-cd49-4f98-b0ad-c890126a898f\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") "
Jul 23 01:01:35.005920 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.005775    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87745493-cd49-4f98-b0ad-c890126a898f-openshift-service-ca\") pod \"87745493-cd49-4f98-b0ad-c890126a898f\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") "
Jul 23 01:01:35.005920 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.005812    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg7cv\" (UniqueName: \"kubernetes.io/projected/87745493-cd49-4f98-b0ad-c890126a898f-kube-api-access-hg7cv\") pod \"87745493-cd49-4f98-b0ad-c890126a898f\" (UID: \"87745493-cd49-4f98-b0ad-c890126a898f\") "
Jul 23 01:01:35.006157 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.006128    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87745493-cd49-4f98-b0ad-c890126a898f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87745493-cd49-4f98-b0ad-c890126a898f" (UID: "87745493-cd49-4f98-b0ad-c890126a898f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:01:35.007857 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.007830    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87745493-cd49-4f98-b0ad-c890126a898f-kube-api-access-hg7cv" (OuterVolumeSpecName: "kube-api-access-hg7cv") pod "87745493-cd49-4f98-b0ad-c890126a898f" (UID: "87745493-cd49-4f98-b0ad-c890126a898f"). InnerVolumeSpecName "kube-api-access-hg7cv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:01:35.007942 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.007874    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87745493-cd49-4f98-b0ad-c890126a898f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87745493-cd49-4f98-b0ad-c890126a898f" (UID: "87745493-cd49-4f98-b0ad-c890126a898f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 01:01:35.106600 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.106565    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87745493-cd49-4f98-b0ad-c890126a898f-maas-api-tls\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:01:35.106600 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.106597    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87745493-cd49-4f98-b0ad-c890126a898f-openshift-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:01:35.106600 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.106609    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg7cv\" (UniqueName: \"kubernetes.io/projected/87745493-cd49-4f98-b0ad-c890126a898f-kube-api-access-hg7cv\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:01:35.619851 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.619816    2561 generic.go:356] "Generic (PLEG): container finished" podID="87745493-cd49-4f98-b0ad-c890126a898f" containerID="ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc" exitCode=0
Jul 23 01:01:35.620060 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.619860    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"
Jul 23 01:01:35.620060 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.619866    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx" event={"ID":"87745493-cd49-4f98-b0ad-c890126a898f","Type":"ContainerDied","Data":"ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc"}
Jul 23 01:01:35.620060 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.619917    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx" event={"ID":"87745493-cd49-4f98-b0ad-c890126a898f","Type":"ContainerDied","Data":"3664fe82dad2765776adbf69e5642879d593a17c24f1030d88a613a3c2f13d63"}
Jul 23 01:01:35.620060 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.619949    2561 scope.go:117] "RemoveContainer" containerID="ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc"
Jul 23 01:01:35.629254 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.629229    2561 scope.go:117] "RemoveContainer" containerID="ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc"
Jul 23 01:01:35.629581 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:01:35.629562    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc\": container with ID starting with ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc not found: ID does not exist" containerID="ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc"
Jul 23 01:01:35.629642 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.629591    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc"} err="failed to get container status \"ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc\": rpc error: code = NotFound desc = could not find container \"ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc\": container with ID starting with ba73ed1e2d63602520ebf15c9f243e15c6fbc07990178e3819a7fcb3b33480fc not found: ID does not exist"
Jul 23 01:01:35.641423 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.641392    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"]
Jul 23 01:01:35.644680 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:35.644656    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0828012d-66cc79cb94-7fpzx"]
Jul 23 01:01:36.787342 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:01:36.787305    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87745493-cd49-4f98-b0ad-c890126a898f" path="/var/lib/kubelet/pods/87745493-cd49-4f98-b0ad-c890126a898f/volumes"
Jul 23 01:02:29.228356 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:29.228318    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"]
Jul 23 01:02:29.228888 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:29.228611    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj" podUID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerName="istio-proxy" containerID="cri-o://05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2" gracePeriod=30
Jul 23 01:02:34.486913 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.486884    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:02:34.627871 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.627771    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-envoy\") pod \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") "
Jul 23 01:02:34.627871 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.627827    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istiod-ca-cert\") pod \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") "
Jul 23 01:02:34.627871 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.627856    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-credential-socket\") pod \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") "
Jul 23 01:02:34.628148 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.627901    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jdc2\" (UniqueName: \"kubernetes.io/projected/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-kube-api-access-4jdc2\") pod \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") "
Jul 23 01:02:34.628148 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.627947    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-workload-certs\") pod \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") "
Jul 23 01:02:34.628148 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.627981    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-data\") pod \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") "
Jul 23 01:02:34.628305 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.628169    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" (UID: "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:02:34.628383 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.628296    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" (UID: "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:02:34.628383 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.628008    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-podinfo\") pod \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") "
Jul 23 01:02:34.628501 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.628348    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" (UID: "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:02:34.628501 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.628444    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-workload-socket\") pod \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") "
Jul 23 01:02:34.628616 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.628521    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-token\") pod \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\" (UID: \"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6\") "
Jul 23 01:02:34.628869 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.628770    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" (UID: "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:02:34.628869 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.628776    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:02:34.628869 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.628820    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:02:34.628869 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.628836    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:02:34.629145 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.629102    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-data" (OuterVolumeSpecName: "istio-data") pod "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" (UID: "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:02:34.630863 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.630781    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-token" (OuterVolumeSpecName: "istio-token") pod "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" (UID: "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:02:34.630863 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.630831    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" (UID: "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 01:02:34.630863 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.630853    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-kube-api-access-4jdc2" (OuterVolumeSpecName: "kube-api-access-4jdc2") pod "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" (UID: "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6"). InnerVolumeSpecName "kube-api-access-4jdc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:02:34.631087 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.630873    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" (UID: "3fa3852c-7e45-4f12-aadd-a9399fc5ebc6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:02:34.729276 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.729238    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:02:34.729276 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.729272    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jdc2\" (UniqueName: \"kubernetes.io/projected/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-kube-api-access-4jdc2\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:02:34.729276 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.729281    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:02:34.729556 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.729290    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:02:34.729556 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.729298    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:02:34.729556 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.729307    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:02:34.896748 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.896657    2561 generic.go:356] "Generic (PLEG): container finished" podID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerID="05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2" exitCode=0
Jul 23 01:02:34.896748 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.896705    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"
Jul 23 01:02:34.896748 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.896704    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj" event={"ID":"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6","Type":"ContainerDied","Data":"05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2"}
Jul 23 01:02:34.897104 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.896764    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj" event={"ID":"3fa3852c-7e45-4f12-aadd-a9399fc5ebc6","Type":"ContainerDied","Data":"e9a073e2461a80ccc6de1b5edae388fd191aa2cc939a0d094d0b2636cf1046aa"}
Jul 23 01:02:34.897104 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.896789    2561 scope.go:117] "RemoveContainer" containerID="05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2"
Jul 23 01:02:34.905530 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.905513    2561 scope.go:117] "RemoveContainer" containerID="809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20"
Jul 23 01:02:34.913889 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.913867    2561 scope.go:117] "RemoveContainer" containerID="05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2"
Jul 23 01:02:34.914217 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:02:34.914195    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2\": container with ID starting with 05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2 not found: ID does not exist" containerID="05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2"
Jul 23 01:02:34.914266 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.914228    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2"} err="failed to get container status \"05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2\": rpc error: code = NotFound desc = could not find container \"05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2\": container with ID starting with 05a4782cb4427fe95f92c0b7512b72ac84e8c40f47985ea2b89da43d18f1d7d2 not found: ID does not exist"
Jul 23 01:02:34.914266 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.914250    2561 scope.go:117] "RemoveContainer" containerID="809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20"
Jul 23 01:02:34.914573 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:02:34.914551    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20\": container with ID starting with 809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20 not found: ID does not exist" containerID="809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20"
Jul 23 01:02:34.914644 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.914579    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20"} err="failed to get container status \"809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20\": rpc error: code = NotFound desc = could not find container \"809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20\": container with ID starting with 809fc4f886f31455522980084cbd999e21b5f2cb71325fdf6c84bb8997de1c20 not found: ID does not exist"
Jul 23 01:02:34.914752 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.914734    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"]
Jul 23 01:02:34.919005 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:34.918982    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0828012d-openshift-default-5dc88bd6f5-swjpj"]
Jul 23 01:02:36.788578 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:02:36.788545    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" path="/var/lib/kubelet/pods/3fa3852c-7e45-4f12-aadd-a9399fc5ebc6/volumes"
Jul 23 01:03:16.833128 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:16.833100    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 01:03:16.835796 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:16.835771    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 01:03:16.941482 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:16.941452    2561 scope.go:117] "RemoveContainer" containerID="89606c2802227cd707012c3e8645e5b7107ec86b567636a5eadeb11c0d33a4cf"
Jul 23 01:03:24.178512 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.178471    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"]
Jul 23 01:03:24.179008 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.178989    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerName="istio-proxy"
Jul 23 01:03:24.179084 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.179011    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerName="istio-proxy"
Jul 23 01:03:24.179084 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.179020    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerName="istio-proxy"
Jul 23 01:03:24.179084 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.179026    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerName="istio-proxy"
Jul 23 01:03:24.179084 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.179044    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87745493-cd49-4f98-b0ad-c890126a898f" containerName="maas-api"
Jul 23 01:03:24.179084 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.179049    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="87745493-cd49-4f98-b0ad-c890126a898f" containerName="maas-api"
Jul 23 01:03:24.179344 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.179114    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="87745493-cd49-4f98-b0ad-c890126a898f" containerName="maas-api"
Jul 23 01:03:24.179344 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.179121    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerName="istio-proxy"
Jul 23 01:03:24.182167 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.182146    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.184774 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.184753    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-ce8306-openshift-default-dockercfg-9ttf2\""
Jul 23 01:03:24.197034 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.197008    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"]
Jul 23 01:03:24.361555 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.361493    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-istio-envoy\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.361555 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.361543    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-workload-certs\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.361830 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.361637    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fe95d38-5a55-436c-9d7f-150244233a44-istio-podinfo\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.361830 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.361684    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fe95d38-5a55-436c-9d7f-150244233a44-istiod-ca-cert\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.361830 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.361715    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fe95d38-5a55-436c-9d7f-150244233a44-istio-token\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.361830 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.361732    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt54b\" (UniqueName: \"kubernetes.io/projected/5fe95d38-5a55-436c-9d7f-150244233a44-kube-api-access-vt54b\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.361830 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.361799    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-credential-socket\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.362014 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.361837    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-istio-data\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.362014 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.361870    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-workload-socket\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.462557 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.462524    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fe95d38-5a55-436c-9d7f-150244233a44-istio-token\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.462557 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.462558    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt54b\" (UniqueName: \"kubernetes.io/projected/5fe95d38-5a55-436c-9d7f-150244233a44-kube-api-access-vt54b\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.462779 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.462590    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-credential-socket\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.462779 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.462694    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-istio-data\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.462779 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.462748    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-workload-socket\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.462936 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.462783    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-istio-envoy\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.462936 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.462821    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-workload-certs\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.462936 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.462863    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fe95d38-5a55-436c-9d7f-150244233a44-istio-podinfo\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.462936 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.462894    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fe95d38-5a55-436c-9d7f-150244233a44-istiod-ca-cert\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.462936 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.462900    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-credential-socket\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.463205 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.463091    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-istio-data\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.463205 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.463142    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-workload-socket\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.463205 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.463176    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-workload-certs\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.463608 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.463585    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fe95d38-5a55-436c-9d7f-150244233a44-istiod-ca-cert\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.464995 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.464967    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-istio-envoy\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.465268 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.465249    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fe95d38-5a55-436c-9d7f-150244233a44-istio-podinfo\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.470238 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.470216    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt54b\" (UniqueName: \"kubernetes.io/projected/5fe95d38-5a55-436c-9d7f-150244233a44-kube-api-access-vt54b\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.470353 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.470339    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fe95d38-5a55-436c-9d7f-150244233a44-istio-token\") pod \"e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") " pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.491492 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.491469    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:24.624795 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.624769    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"]
Jul 23 01:03:24.626273 ip-10-0-133-215 kubenswrapper[2561]: W0723 01:03:24.626241    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe95d38_5a55_436c_9d7f_150244233a44.slice/crio-b6884a23569b736aaa876b6bd632e9708315a59ac8df90e9ca3605d2e5371ab6 WatchSource:0}: Error finding container b6884a23569b736aaa876b6bd632e9708315a59ac8df90e9ca3605d2e5371ab6: Status 404 returned error can't find the container with id b6884a23569b736aaa876b6bd632e9708315a59ac8df90e9ca3605d2e5371ab6
Jul 23 01:03:24.628721 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.628681    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:03:24.628816 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.628767    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:03:24.628873 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:24.628824    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:03:25.126680 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:25.126633    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" event={"ID":"5fe95d38-5a55-436c-9d7f-150244233a44","Type":"ContainerStarted","Data":"b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe"}
Jul 23 01:03:25.126887 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:25.126693    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" event={"ID":"5fe95d38-5a55-436c-9d7f-150244233a44","Type":"ContainerStarted","Data":"b6884a23569b736aaa876b6bd632e9708315a59ac8df90e9ca3605d2e5371ab6"}
Jul 23 01:03:25.153657 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:25.153601    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" podStartSLOduration=1.153583531 podStartE2EDuration="1.153583531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:03:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:03:25.151670543 +0000 UTC m=+2408.958081789" watchObservedRunningTime="2026-07-23 01:03:25.153583531 +0000 UTC m=+2408.959994764"
Jul 23 01:03:25.492312 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:25.492280    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:25.497226 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:25.497204    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:26.132975 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:26.132945    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:26.133944 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:26.133923    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:27.397834 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.397804    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"]
Jul 23 01:03:27.398214 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.398200    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fa3852c-7e45-4f12-aadd-a9399fc5ebc6" containerName="istio-proxy"
Jul 23 01:03:27.401123 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.401108    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:27.403387 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.403337    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-ce8306-serving-cert\""
Jul 23 01:03:27.412959 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.412934    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"]
Jul 23 01:03:27.588715 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.588675    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z\" (UID: \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:27.588901 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.588737    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fxm9\" (UniqueName: \"kubernetes.io/projected/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-kube-api-access-8fxm9\") pod \"maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z\" (UID: \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:27.588901 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.588834    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z\" (UID: \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:27.689759 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.689725    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fxm9\" (UniqueName: \"kubernetes.io/projected/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-kube-api-access-8fxm9\") pod \"maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z\" (UID: \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:27.689921 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.689792    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z\" (UID: \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:27.689921 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.689832    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z\" (UID: \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:27.690500 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.690480    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z\" (UID: \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:27.692136 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.692112    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z\" (UID: \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:27.701087 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.701067    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fxm9\" (UniqueName: \"kubernetes.io/projected/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-kube-api-access-8fxm9\") pod \"maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z\" (UID: \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:27.715942 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.715915    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:27.849602 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:27.849573    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"]
Jul 23 01:03:27.852098 ip-10-0-133-215 kubenswrapper[2561]: W0723 01:03:27.852073    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6380bdc0_7c0e_4f24_9c2c_20b97415eec2.slice/crio-8cca02ee7d8f8305a77a519e589218d6d5fb531e8e446104ab1935911d0d254d WatchSource:0}: Error finding container 8cca02ee7d8f8305a77a519e589218d6d5fb531e8e446104ab1935911d0d254d: Status 404 returned error can't find the container with id 8cca02ee7d8f8305a77a519e589218d6d5fb531e8e446104ab1935911d0d254d
Jul 23 01:03:28.143060 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:28.142973    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z" event={"ID":"6380bdc0-7c0e-4f24-9c2c-20b97415eec2","Type":"ContainerStarted","Data":"8cca02ee7d8f8305a77a519e589218d6d5fb531e8e446104ab1935911d0d254d"}
Jul 23 01:03:29.153720 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:29.153682    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z" event={"ID":"6380bdc0-7c0e-4f24-9c2c-20b97415eec2","Type":"ContainerStarted","Data":"3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2"}
Jul 23 01:03:29.154133 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:29.153796    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:29.171120 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:29.171066    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z" podStartSLOduration=1.6571819429999999 podStartE2EDuration="2.171051418s" totalImagesPullingTime="513.869475ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:03:29.169949672 +0000 UTC m=+2412.976360906" watchObservedRunningTime="2026-07-23 01:03:29.171051418 +0000 UTC m=+2412.977462651"
Jul 23 01:03:32.230353 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.230319    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"]
Jul 23 01:03:32.234840 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.234816    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.237341 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.237317    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-bb9f10-openshift-default-dockercfg-pj8v6\""
Jul 23 01:03:32.245940 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.245916    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"]
Jul 23 01:03:32.331447 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.331407    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-data\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.331642 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.331461    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6be27c27-63d6-489c-952b-bd6f1d8370d5-istiod-ca-cert\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.331642 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.331513    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-workload-certs\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.331642 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.331548    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-token\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.331642 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.331617    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8v7l\" (UniqueName: \"kubernetes.io/projected/6be27c27-63d6-489c-952b-bd6f1d8370d5-kube-api-access-p8v7l\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.331868 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.331667    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-envoy\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.331868 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.331726    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-workload-socket\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.331868 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.331753    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-podinfo\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.331868 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.331795    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-credential-socket\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.432355 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.432298    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6be27c27-63d6-489c-952b-bd6f1d8370d5-istiod-ca-cert\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.432355 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.432384    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-workload-certs\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.432809 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.432425    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-token\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.432809 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.432483    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8v7l\" (UniqueName: \"kubernetes.io/projected/6be27c27-63d6-489c-952b-bd6f1d8370d5-kube-api-access-p8v7l\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.432809 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.432520    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-envoy\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.432809 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.432573    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-workload-socket\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.432809 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.432599    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-podinfo\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.432809 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.432641    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-credential-socket\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.432809 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.432691    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-data\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.433186 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.433110    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-data\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.433974 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.433944    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6be27c27-63d6-489c-952b-bd6f1d8370d5-istiod-ca-cert\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.434217 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.434194    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-workload-certs\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.435313 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.435289    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-workload-socket\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.435313 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.435303    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-credential-socket\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.436905 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.436882    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-envoy\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.437416 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.437396    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-podinfo\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.446307 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.446282    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-token\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.446405 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.446346    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8v7l\" (UniqueName: \"kubernetes.io/projected/6be27c27-63d6-489c-952b-bd6f1d8370d5-kube-api-access-p8v7l\") pod \"e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") " pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.548477 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.548356    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:32.683140 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.683115    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"]
Jul 23 01:03:32.685258 ip-10-0-133-215 kubenswrapper[2561]: W0723 01:03:32.685228    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be27c27_63d6_489c_952b_bd6f1d8370d5.slice/crio-168a7b85dbcaaee862958e6a8605e2359b5088b8567916d01c62cc2c295fec1f WatchSource:0}: Error finding container 168a7b85dbcaaee862958e6a8605e2359b5088b8567916d01c62cc2c295fec1f: Status 404 returned error can't find the container with id 168a7b85dbcaaee862958e6a8605e2359b5088b8567916d01c62cc2c295fec1f
Jul 23 01:03:32.687312 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.687283    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:03:32.687394 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.687345    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:03:32.687443 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:32.687404    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:03:33.174336 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:33.174292    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" event={"ID":"6be27c27-63d6-489c-952b-bd6f1d8370d5","Type":"ContainerStarted","Data":"1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86"}
Jul 23 01:03:33.174535 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:33.174358    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" event={"ID":"6be27c27-63d6-489c-952b-bd6f1d8370d5","Type":"ContainerStarted","Data":"168a7b85dbcaaee862958e6a8605e2359b5088b8567916d01c62cc2c295fec1f"}
Jul 23 01:03:33.220423 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:33.220355    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" podStartSLOduration=1.2203406509999999 podStartE2EDuration="1.220340651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:03:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:03:33.218379148 +0000 UTC m=+2417.024790393" watchObservedRunningTime="2026-07-23 01:03:33.220340651 +0000 UTC m=+2417.026751883"
Jul 23 01:03:33.548908 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:33.548874    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:33.553787 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:33.553759    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:34.178632 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:34.178601    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:34.179725 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:34.179706    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:35.164935 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:35.164903    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:03:41.134285 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:41.134248    2561 patch_prober.go:28] interesting pod/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 23 01:03:41.134905 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:41.134315    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 23 01:03:41.219629 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:41.219599    2561 generic.go:356] "Generic (PLEG): container finished" podID="5fe95d38-5a55-436c-9d7f-150244233a44" containerID="b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe" exitCode=0
Jul 23 01:03:41.219812 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:41.219661    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" event={"ID":"5fe95d38-5a55-436c-9d7f-150244233a44","Type":"ContainerDied","Data":"b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe"}
Jul 23 01:03:41.220041 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:41.220025    2561 scope.go:117] "RemoveContainer" containerID="b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe"
Jul 23 01:03:41.221105 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:41.221073    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:03:41.221205 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:41.221140    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:03:41.221205 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:41.221170    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:03:42.233299 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:42.233253    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" event={"ID":"5fe95d38-5a55-436c-9d7f-150244233a44","Type":"ContainerStarted","Data":"d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8"}
Jul 23 01:03:42.492424 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:42.492312    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:42.492584 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:42.492447    2561 patch_prober.go:28] interesting pod/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 23 01:03:42.492584 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:42.492483    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 23 01:03:43.492769 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:43.492729    2561 patch_prober.go:28] interesting pod/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 23 01:03:43.493180 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:43.492801    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 23 01:03:44.492181 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:44.492138    2561 patch_prober.go:28] interesting pod/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 23 01:03:44.492395 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:44.492193    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 23 01:03:45.492705 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:45.492666    2561 patch_prober.go:28] interesting pod/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 23 01:03:45.493107 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:45.492729    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 23 01:03:46.496325 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:46.496292    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:46.496820 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:46.496779    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:46.497613 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:46.497591    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:03:49.180627 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:49.180577    2561 patch_prober.go:28] interesting pod/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 23 01:03:49.181126 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:49.180653    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 23 01:03:49.285840 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:49.285801    2561 generic.go:356] "Generic (PLEG): container finished" podID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerID="1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86" exitCode=0
Jul 23 01:03:49.286044 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:49.285846    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" event={"ID":"6be27c27-63d6-489c-952b-bd6f1d8370d5","Type":"ContainerDied","Data":"1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86"}
Jul 23 01:03:49.286298 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:49.286279    2561 scope.go:117] "RemoveContainer" containerID="1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86"
Jul 23 01:03:49.287455 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:49.287424    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:03:49.287583 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:49.287509    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:03:49.287583 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:49.287557    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:03:50.292253 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:50.292219    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" event={"ID":"6be27c27-63d6-489c-952b-bd6f1d8370d5","Type":"ContainerStarted","Data":"1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22"}
Jul 23 01:03:50.549570 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:50.549469    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:50.549848 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:50.549542    2561 patch_prober.go:28] interesting pod/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 23 01:03:50.550028 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:50.550001    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 23 01:03:51.549376 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:51.549312    2561 patch_prober.go:28] interesting pod/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 23 01:03:51.549906 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:51.549403    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 23 01:03:52.549352 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:52.549314    2561 patch_prober.go:28] interesting pod/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 23 01:03:52.549673 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:52.549646    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 23 01:03:53.549648 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:53.549605    2561 patch_prober.go:28] interesting pod/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 23 01:03:53.549862 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:53.549675    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 23 01:03:54.552826 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:54.552793    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:54.553284 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:54.553060    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:03:54.553733 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:03:54.553717    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:04:16.967322 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:04:16.967290    2561 scope.go:117] "RemoveContainer" containerID="b2b433ab0ef98f0b885571e014dc803671ef57f393ac3e5b5cc4b975601e179c"
Jul 23 01:05:26.285964 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.285881    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"]
Jul 23 01:05:26.286515 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.286133    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z" podUID="6380bdc0-7c0e-4f24-9c2c-20b97415eec2" containerName="maas-api" containerID="cri-o://3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2" gracePeriod=30
Jul 23 01:05:26.543888 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.543833    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:05:26.654384 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.654323    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-openshift-service-ca\") pod \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\" (UID: \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\") "
Jul 23 01:05:26.654557 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.654413    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fxm9\" (UniqueName: \"kubernetes.io/projected/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-kube-api-access-8fxm9\") pod \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\" (UID: \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\") "
Jul 23 01:05:26.654557 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.654488    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-maas-api-tls\") pod \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\" (UID: \"6380bdc0-7c0e-4f24-9c2c-20b97415eec2\") "
Jul 23 01:05:26.654769 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.654737    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6380bdc0-7c0e-4f24-9c2c-20b97415eec2" (UID: "6380bdc0-7c0e-4f24-9c2c-20b97415eec2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:05:26.656582 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.656546    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6380bdc0-7c0e-4f24-9c2c-20b97415eec2" (UID: "6380bdc0-7c0e-4f24-9c2c-20b97415eec2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 01:05:26.656712 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.656617    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-kube-api-access-8fxm9" (OuterVolumeSpecName: "kube-api-access-8fxm9") pod "6380bdc0-7c0e-4f24-9c2c-20b97415eec2" (UID: "6380bdc0-7c0e-4f24-9c2c-20b97415eec2"). InnerVolumeSpecName "kube-api-access-8fxm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:05:26.751962 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.751931    2561 generic.go:356] "Generic (PLEG): container finished" podID="6380bdc0-7c0e-4f24-9c2c-20b97415eec2" containerID="3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2" exitCode=0
Jul 23 01:05:26.752135 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.751972    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"
Jul 23 01:05:26.752135 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.752003    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z" event={"ID":"6380bdc0-7c0e-4f24-9c2c-20b97415eec2","Type":"ContainerDied","Data":"3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2"}
Jul 23 01:05:26.752135 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.752043    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z" event={"ID":"6380bdc0-7c0e-4f24-9c2c-20b97415eec2","Type":"ContainerDied","Data":"8cca02ee7d8f8305a77a519e589218d6d5fb531e8e446104ab1935911d0d254d"}
Jul 23 01:05:26.752135 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.752065    2561 scope.go:117] "RemoveContainer" containerID="3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2"
Jul 23 01:05:26.755111 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.755089    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-maas-api-tls\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:05:26.755111 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.755111    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-openshift-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:05:26.755277 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.755121    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fxm9\" (UniqueName: \"kubernetes.io/projected/6380bdc0-7c0e-4f24-9c2c-20b97415eec2-kube-api-access-8fxm9\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:05:26.761279 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.761260    2561 scope.go:117] "RemoveContainer" containerID="3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2"
Jul 23 01:05:26.761565 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:05:26.761544    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2\": container with ID starting with 3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2 not found: ID does not exist" containerID="3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2"
Jul 23 01:05:26.761621 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.761576    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2"} err="failed to get container status \"3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2\": rpc error: code = NotFound desc = could not find container \"3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2\": container with ID starting with 3b66e28b85f38ef67e926dd57994a2109b5544a1a5fe64096ac98f3aba8930c2 not found: ID does not exist"
Jul 23 01:05:26.773559 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.773539    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"]
Jul 23 01:05:26.779501 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.779479    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce8306-5849999bdd-gd52z"]
Jul 23 01:05:26.787529 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:26.787506    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6380bdc0-7c0e-4f24-9c2c-20b97415eec2" path="/var/lib/kubelet/pods/6380bdc0-7c0e-4f24-9c2c-20b97415eec2/volumes"
Jul 23 01:05:37.528142 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:37.528108    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"]
Jul 23 01:05:37.528637 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:37.528400    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" containerName="istio-proxy" containerID="cri-o://d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8" gracePeriod=30
Jul 23 01:05:42.779664 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.779638    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:05:42.828603 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.828511    2561 generic.go:356] "Generic (PLEG): container finished" podID="5fe95d38-5a55-436c-9d7f-150244233a44" containerID="d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8" exitCode=0
Jul 23 01:05:42.828603 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.828575    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"
Jul 23 01:05:42.828840 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.828610    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" event={"ID":"5fe95d38-5a55-436c-9d7f-150244233a44","Type":"ContainerDied","Data":"d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8"}
Jul 23 01:05:42.828840 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.828714    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd" event={"ID":"5fe95d38-5a55-436c-9d7f-150244233a44","Type":"ContainerDied","Data":"b6884a23569b736aaa876b6bd632e9708315a59ac8df90e9ca3605d2e5371ab6"}
Jul 23 01:05:42.828840 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.828755    2561 scope.go:117] "RemoveContainer" containerID="d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8"
Jul 23 01:05:42.837398 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.837382    2561 scope.go:117] "RemoveContainer" containerID="b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe"
Jul 23 01:05:42.845559 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.845528    2561 scope.go:117] "RemoveContainer" containerID="d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8"
Jul 23 01:05:42.845902 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:05:42.845875    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8\": container with ID starting with d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8 not found: ID does not exist" containerID="d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8"
Jul 23 01:05:42.845996 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.845912    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8"} err="failed to get container status \"d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8\": rpc error: code = NotFound desc = could not find container \"d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8\": container with ID starting with d6a72f555b11caccd9cc3e7de68c441df9a6da6e2c0e79afe5edb6f5386504f8 not found: ID does not exist"
Jul 23 01:05:42.845996 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.845937    2561 scope.go:117] "RemoveContainer" containerID="b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe"
Jul 23 01:05:42.846227 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:05:42.846209    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe\": container with ID starting with b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe not found: ID does not exist" containerID="b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe"
Jul 23 01:05:42.846291 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.846234    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe"} err="failed to get container status \"b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe\": rpc error: code = NotFound desc = could not find container \"b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe\": container with ID starting with b339c9d9fd4e27251d9ef764e09cccaa57503b8d546521fe21f2d1c74b4b3bfe not found: ID does not exist"
Jul 23 01:05:42.894575 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.894541    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fe95d38-5a55-436c-9d7f-150244233a44-istio-token\") pod \"5fe95d38-5a55-436c-9d7f-150244233a44\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") "
Jul 23 01:05:42.894575 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.894576    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-workload-socket\") pod \"5fe95d38-5a55-436c-9d7f-150244233a44\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") "
Jul 23 01:05:42.894790 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.894612    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt54b\" (UniqueName: \"kubernetes.io/projected/5fe95d38-5a55-436c-9d7f-150244233a44-kube-api-access-vt54b\") pod \"5fe95d38-5a55-436c-9d7f-150244233a44\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") "
Jul 23 01:05:42.894790 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.894640    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-workload-certs\") pod \"5fe95d38-5a55-436c-9d7f-150244233a44\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") "
Jul 23 01:05:42.894790 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.894660    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-credential-socket\") pod \"5fe95d38-5a55-436c-9d7f-150244233a44\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") "
Jul 23 01:05:42.894790 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.894720    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fe95d38-5a55-436c-9d7f-150244233a44-istiod-ca-cert\") pod \"5fe95d38-5a55-436c-9d7f-150244233a44\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") "
Jul 23 01:05:42.894790 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.894771    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-istio-envoy\") pod \"5fe95d38-5a55-436c-9d7f-150244233a44\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") "
Jul 23 01:05:42.895054 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.894813    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fe95d38-5a55-436c-9d7f-150244233a44-istio-podinfo\") pod \"5fe95d38-5a55-436c-9d7f-150244233a44\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") "
Jul 23 01:05:42.895054 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.894834    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5fe95d38-5a55-436c-9d7f-150244233a44" (UID: "5fe95d38-5a55-436c-9d7f-150244233a44"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:05:42.895054 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.894866    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-istio-data\") pod \"5fe95d38-5a55-436c-9d7f-150244233a44\" (UID: \"5fe95d38-5a55-436c-9d7f-150244233a44\") "
Jul 23 01:05:42.895213 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.895052    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5fe95d38-5a55-436c-9d7f-150244233a44" (UID: "5fe95d38-5a55-436c-9d7f-150244233a44"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:05:42.895213 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.895056    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5fe95d38-5a55-436c-9d7f-150244233a44" (UID: "5fe95d38-5a55-436c-9d7f-150244233a44"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:05:42.895320 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.895213    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:05:42.895320 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.895234    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:05:42.895320 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.895248    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:05:42.895320 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.895171    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fe95d38-5a55-436c-9d7f-150244233a44-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5fe95d38-5a55-436c-9d7f-150244233a44" (UID: "5fe95d38-5a55-436c-9d7f-150244233a44"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:05:42.895949 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.895922    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-istio-data" (OuterVolumeSpecName: "istio-data") pod "5fe95d38-5a55-436c-9d7f-150244233a44" (UID: "5fe95d38-5a55-436c-9d7f-150244233a44"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:05:42.896915 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.896891    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fe95d38-5a55-436c-9d7f-150244233a44-kube-api-access-vt54b" (OuterVolumeSpecName: "kube-api-access-vt54b") pod "5fe95d38-5a55-436c-9d7f-150244233a44" (UID: "5fe95d38-5a55-436c-9d7f-150244233a44"). InnerVolumeSpecName "kube-api-access-vt54b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:05:42.897053 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.897032    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5fe95d38-5a55-436c-9d7f-150244233a44" (UID: "5fe95d38-5a55-436c-9d7f-150244233a44"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:05:42.897118 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.897101    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fe95d38-5a55-436c-9d7f-150244233a44-istio-token" (OuterVolumeSpecName: "istio-token") pod "5fe95d38-5a55-436c-9d7f-150244233a44" (UID: "5fe95d38-5a55-436c-9d7f-150244233a44"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:05:42.897278 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.897257    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5fe95d38-5a55-436c-9d7f-150244233a44-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5fe95d38-5a55-436c-9d7f-150244233a44" (UID: "5fe95d38-5a55-436c-9d7f-150244233a44"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 01:05:42.995797 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.995765    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:05:42.995797 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.995791    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fe95d38-5a55-436c-9d7f-150244233a44-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:05:42.995797 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.995800    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fe95d38-5a55-436c-9d7f-150244233a44-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:05:42.996010 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.995810    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fe95d38-5a55-436c-9d7f-150244233a44-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:05:42.996010 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.995819    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt54b\" (UniqueName: \"kubernetes.io/projected/5fe95d38-5a55-436c-9d7f-150244233a44-kube-api-access-vt54b\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:05:42.996010 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:42.995828    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fe95d38-5a55-436c-9d7f-150244233a44-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:05:43.152313 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:43.152275    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"]
Jul 23 01:05:43.158730 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:43.158706    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-ce8306-openshift-default-5c4b78d8cf-xvxkd"]
Jul 23 01:05:44.792170 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:44.792134    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" path="/var/lib/kubelet/pods/5fe95d38-5a55-436c-9d7f-150244233a44/volumes"
Jul 23 01:05:59.686481 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:59.686445    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"]
Jul 23 01:05:59.687116 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:05:59.686704    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerName="istio-proxy" containerID="cri-o://1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22" gracePeriod=30
Jul 23 01:06:00.576816 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.576775    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"]
Jul 23 01:06:00.577198 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.577180    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" containerName="istio-proxy"
Jul 23 01:06:00.577290 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.577201    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" containerName="istio-proxy"
Jul 23 01:06:00.577290 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.577213    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" containerName="istio-proxy"
Jul 23 01:06:00.577290 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.577221    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" containerName="istio-proxy"
Jul 23 01:06:00.577290 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.577235    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6380bdc0-7c0e-4f24-9c2c-20b97415eec2" containerName="maas-api"
Jul 23 01:06:00.577290 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.577243    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6380bdc0-7c0e-4f24-9c2c-20b97415eec2" containerName="maas-api"
Jul 23 01:06:00.577568 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.577324    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" containerName="istio-proxy"
Jul 23 01:06:00.577568 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.577336    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6380bdc0-7c0e-4f24-9c2c-20b97415eec2" containerName="maas-api"
Jul 23 01:06:00.579345 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.579322    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.582280 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.582260    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c2ab89-openshift-default-dockercfg-5mdvh\""
Jul 23 01:06:00.594694 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.594668    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"]
Jul 23 01:06:00.745869 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.745837    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-data\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.745869 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.745875    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istiod-ca-cert\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.746306 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.745896    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-token\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.746306 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.745989    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-envoy\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.746306 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.746020    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-credential-socket\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.746306 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.746057    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-podinfo\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.746306 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.746081    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mdht\" (UniqueName: \"kubernetes.io/projected/814a5df0-5cd6-4f29-91c0-e88fa26b8490-kube-api-access-2mdht\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.746306 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.746103    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-workload-socket\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.746306 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.746135    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-workload-certs\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.847083 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.846993    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-envoy\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.847083 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.847041    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-credential-socket\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.847288 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.847173    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-podinfo\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.847288 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.847220    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mdht\" (UniqueName: \"kubernetes.io/projected/814a5df0-5cd6-4f29-91c0-e88fa26b8490-kube-api-access-2mdht\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.847288 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.847258    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-workload-socket\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.847464 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.847296    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-workload-certs\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.847464 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.847334    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-data\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.847464 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.847397    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-credential-socket\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.847464 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.847401    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istiod-ca-cert\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.847679 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.847465    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-token\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.847679 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.847640    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-workload-socket\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.847796 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.847751    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-data\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.848026 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.848004    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-workload-certs\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.848128 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.848106    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istiod-ca-cert\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.849428 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.849407    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-envoy\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.849705 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.849688    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-podinfo\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.856789 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.856766    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-token\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.856924 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.856882    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mdht\" (UniqueName: \"kubernetes.io/projected/814a5df0-5cd6-4f29-91c0-e88fa26b8490-kube-api-access-2mdht\") pod \"e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") " pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:00.889254 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:00.889230    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:01.033475 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:01.033442    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"]
Jul 23 01:06:01.035321 ip-10-0-133-215 kubenswrapper[2561]: W0723 01:06:01.035295    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod814a5df0_5cd6_4f29_91c0_e88fa26b8490.slice/crio-85ebee85a667cd161d235be43c742f263adbdccddcea543d9d582079e736e17b WatchSource:0}: Error finding container 85ebee85a667cd161d235be43c742f263adbdccddcea543d9d582079e736e17b: Status 404 returned error can't find the container with id 85ebee85a667cd161d235be43c742f263adbdccddcea543d9d582079e736e17b
Jul 23 01:06:01.037632 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:01.037602    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:06:01.037719 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:01.037669    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:06:01.037719 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:01.037698    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:06:01.918621 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:01.918572    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" event={"ID":"814a5df0-5cd6-4f29-91c0-e88fa26b8490","Type":"ContainerStarted","Data":"9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827"}
Jul 23 01:06:01.919011 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:01.918665    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" event={"ID":"814a5df0-5cd6-4f29-91c0-e88fa26b8490","Type":"ContainerStarted","Data":"85ebee85a667cd161d235be43c742f263adbdccddcea543d9d582079e736e17b"}
Jul 23 01:06:01.942718 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:01.942667    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" podStartSLOduration=1.942651271 podStartE2EDuration="1.942651271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:06:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:06:01.940795334 +0000 UTC m=+2565.747206567" watchObservedRunningTime="2026-07-23 01:06:01.942651271 +0000 UTC m=+2565.749062504"
Jul 23 01:06:02.890034 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:02.890001    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:02.894974 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:02.894950    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:02.923907 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:02.923881    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:02.924883 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:02.924867    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:03.951717 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:03.951684    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"]
Jul 23 01:06:03.952273 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:03.952254    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fe95d38-5a55-436c-9d7f-150244233a44" containerName="istio-proxy"
Jul 23 01:06:03.954532 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:03.954512    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:03.957456 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:03.957436    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c2ab89-serving-cert\""
Jul 23 01:06:03.967511 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:03.967486    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"]
Jul 23 01:06:03.980818 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:03.980750    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15f94540-dcc5-4c67-8584-553850c92f39-maas-api-tls\") pod \"maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:03.980966 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:03.980846    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns5v6\" (UniqueName: \"kubernetes.io/projected/15f94540-dcc5-4c67-8584-553850c92f39-kube-api-access-ns5v6\") pod \"maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:03.980966 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:03.980895    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f94540-dcc5-4c67-8584-553850c92f39-openshift-service-ca\") pod \"maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:04.081722 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.081681    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15f94540-dcc5-4c67-8584-553850c92f39-maas-api-tls\") pod \"maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:04.081927 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.081737    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns5v6\" (UniqueName: \"kubernetes.io/projected/15f94540-dcc5-4c67-8584-553850c92f39-kube-api-access-ns5v6\") pod \"maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:04.081927 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.081765    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f94540-dcc5-4c67-8584-553850c92f39-openshift-service-ca\") pod \"maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:04.081927 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:06:04.081833    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-serving-cert: secret "maas-api-e2e-api-a-c2ab89-serving-cert" not found
Jul 23 01:06:04.081927 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:06:04.081912    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15f94540-dcc5-4c67-8584-553850c92f39-maas-api-tls podName:15f94540-dcc5-4c67-8584-553850c92f39 nodeName:}" failed. No retries permitted until 2026-07-23 01:06:04.581894694 +0000 UTC m=+2568.388305905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/15f94540-dcc5-4c67-8584-553850c92f39-maas-api-tls") pod "maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6" (UID: "15f94540-dcc5-4c67-8584-553850c92f39") : secret "maas-api-e2e-api-a-c2ab89-serving-cert" not found
Jul 23 01:06:04.082377 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.082343    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f94540-dcc5-4c67-8584-553850c92f39-openshift-service-ca\") pod \"maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:04.093442 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.093419    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns5v6\" (UniqueName: \"kubernetes.io/projected/15f94540-dcc5-4c67-8584-553850c92f39-kube-api-access-ns5v6\") pod \"maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:04.586793 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.586754    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15f94540-dcc5-4c67-8584-553850c92f39-maas-api-tls\") pod \"maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:04.589338 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.589316    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15f94540-dcc5-4c67-8584-553850c92f39-maas-api-tls\") pod \"maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:04.865746 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.865711    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:04.934780 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.934476    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:06:04.938860 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.938010    2561 generic.go:356] "Generic (PLEG): container finished" podID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerID="1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22" exitCode=0
Jul 23 01:06:04.938860 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.938788    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"
Jul 23 01:06:04.939705 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.938781    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" event={"ID":"6be27c27-63d6-489c-952b-bd6f1d8370d5","Type":"ContainerDied","Data":"1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22"}
Jul 23 01:06:04.939705 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.939005    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s" event={"ID":"6be27c27-63d6-489c-952b-bd6f1d8370d5","Type":"ContainerDied","Data":"168a7b85dbcaaee862958e6a8605e2359b5088b8567916d01c62cc2c295fec1f"}
Jul 23 01:06:04.939705 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.939039    2561 scope.go:117] "RemoveContainer" containerID="1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22"
Jul 23 01:06:04.950457 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.950431    2561 scope.go:117] "RemoveContainer" containerID="1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86"
Jul 23 01:06:04.959722 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.959642    2561 scope.go:117] "RemoveContainer" containerID="1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22"
Jul 23 01:06:04.960644 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:06:04.959982    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22\": container with ID starting with 1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22 not found: ID does not exist" containerID="1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22"
Jul 23 01:06:04.960644 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.960485    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22"} err="failed to get container status \"1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22\": rpc error: code = NotFound desc = could not find container \"1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22\": container with ID starting with 1cee7539e71232bffe8c5bd820a05a70faefc856d5c4ca9f46aefe51031e2c22 not found: ID does not exist"
Jul 23 01:06:04.960644 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.960513    2561 scope.go:117] "RemoveContainer" containerID="1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86"
Jul 23 01:06:04.961026 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:06:04.960999    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86\": container with ID starting with 1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86 not found: ID does not exist" containerID="1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86"
Jul 23 01:06:04.961109 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.961037    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86"} err="failed to get container status \"1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86\": rpc error: code = NotFound desc = could not find container \"1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86\": container with ID starting with 1bd0a8c89d91459bfb3ec66c64a24fa66c7a06a34ce57c4e0e2d68fc518dec86 not found: ID does not exist"
Jul 23 01:06:04.990326 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990296    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8v7l\" (UniqueName: \"kubernetes.io/projected/6be27c27-63d6-489c-952b-bd6f1d8370d5-kube-api-access-p8v7l\") pod \"6be27c27-63d6-489c-952b-bd6f1d8370d5\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") "
Jul 23 01:06:04.990518 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990377    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6be27c27-63d6-489c-952b-bd6f1d8370d5-istiod-ca-cert\") pod \"6be27c27-63d6-489c-952b-bd6f1d8370d5\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") "
Jul 23 01:06:04.990518 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990427    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-workload-socket\") pod \"6be27c27-63d6-489c-952b-bd6f1d8370d5\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") "
Jul 23 01:06:04.990518 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990453    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-token\") pod \"6be27c27-63d6-489c-952b-bd6f1d8370d5\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") "
Jul 23 01:06:04.990518 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990482    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-workload-certs\") pod \"6be27c27-63d6-489c-952b-bd6f1d8370d5\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") "
Jul 23 01:06:04.990518 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990512    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-credential-socket\") pod \"6be27c27-63d6-489c-952b-bd6f1d8370d5\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") "
Jul 23 01:06:04.990808 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990537    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-envoy\") pod \"6be27c27-63d6-489c-952b-bd6f1d8370d5\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") "
Jul 23 01:06:04.990808 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990564    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-data\") pod \"6be27c27-63d6-489c-952b-bd6f1d8370d5\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") "
Jul 23 01:06:04.990808 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990592    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-podinfo\") pod \"6be27c27-63d6-489c-952b-bd6f1d8370d5\" (UID: \"6be27c27-63d6-489c-952b-bd6f1d8370d5\") "
Jul 23 01:06:04.990808 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990718    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6be27c27-63d6-489c-952b-bd6f1d8370d5" (UID: "6be27c27-63d6-489c-952b-bd6f1d8370d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:06:04.990808 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990801    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6be27c27-63d6-489c-952b-bd6f1d8370d5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6be27c27-63d6-489c-952b-bd6f1d8370d5" (UID: "6be27c27-63d6-489c-952b-bd6f1d8370d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:06:04.991159 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990995    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6be27c27-63d6-489c-952b-bd6f1d8370d5-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:04.991159 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.991015    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:04.991159 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.990991    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6be27c27-63d6-489c-952b-bd6f1d8370d5" (UID: "6be27c27-63d6-489c-952b-bd6f1d8370d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:06:04.991159 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.991068    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6be27c27-63d6-489c-952b-bd6f1d8370d5" (UID: "6be27c27-63d6-489c-952b-bd6f1d8370d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:06:04.991954 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.991928    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-data" (OuterVolumeSpecName: "istio-data") pod "6be27c27-63d6-489c-952b-bd6f1d8370d5" (UID: "6be27c27-63d6-489c-952b-bd6f1d8370d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:06:04.993251 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.993219    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6be27c27-63d6-489c-952b-bd6f1d8370d5" (UID: "6be27c27-63d6-489c-952b-bd6f1d8370d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 01:06:04.993409 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.993388    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be27c27-63d6-489c-952b-bd6f1d8370d5-kube-api-access-p8v7l" (OuterVolumeSpecName: "kube-api-access-p8v7l") pod "6be27c27-63d6-489c-952b-bd6f1d8370d5" (UID: "6be27c27-63d6-489c-952b-bd6f1d8370d5"). InnerVolumeSpecName "kube-api-access-p8v7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:06:04.993475 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.993421    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-token" (OuterVolumeSpecName: "istio-token") pod "6be27c27-63d6-489c-952b-bd6f1d8370d5" (UID: "6be27c27-63d6-489c-952b-bd6f1d8370d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:06:04.994334 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:04.994310    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6be27c27-63d6-489c-952b-bd6f1d8370d5" (UID: "6be27c27-63d6-489c-952b-bd6f1d8370d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:06:05.014470 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.014445    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"]
Jul 23 01:06:05.015789 ip-10-0-133-215 kubenswrapper[2561]: W0723 01:06:05.015765    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f94540_dcc5_4c67_8584_553850c92f39.slice/crio-ba266b61b392a489c650671ad47f8b69a992a66d1996a2d611afb9f4c7ec7784 WatchSource:0}: Error finding container ba266b61b392a489c650671ad47f8b69a992a66d1996a2d611afb9f4c7ec7784: Status 404 returned error can't find the container with id ba266b61b392a489c650671ad47f8b69a992a66d1996a2d611afb9f4c7ec7784
Jul 23 01:06:05.017267 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.017247    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 01:06:05.091769 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.091735    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8v7l\" (UniqueName: \"kubernetes.io/projected/6be27c27-63d6-489c-952b-bd6f1d8370d5-kube-api-access-p8v7l\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:05.091769 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.091766    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:05.091769 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.091776    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:05.091997 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.091784    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:05.091997 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.091792    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:05.091997 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.091800    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:05.091997 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.091806    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6be27c27-63d6-489c-952b-bd6f1d8370d5-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:05.260776 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.260743    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"]
Jul 23 01:06:05.263980 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.263947    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-bb9f10-openshift-default-7bf575ccdc-wnm9s"]
Jul 23 01:06:05.945706 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.945667    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6" event={"ID":"15f94540-dcc5-4c67-8584-553850c92f39","Type":"ContainerStarted","Data":"0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1"}
Jul 23 01:06:05.945892 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.945720    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6" event={"ID":"15f94540-dcc5-4c67-8584-553850c92f39","Type":"ContainerStarted","Data":"ba266b61b392a489c650671ad47f8b69a992a66d1996a2d611afb9f4c7ec7784"}
Jul 23 01:06:05.945892 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.945774    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:05.962457 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:05.962407    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6" podStartSLOduration=2.463878282 podStartE2EDuration="2.962391476s" totalImagesPullingTime="498.513194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:06:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:06:05.960549884 +0000 UTC m=+2569.766961117" watchObservedRunningTime="2026-07-23 01:06:05.962391476 +0000 UTC m=+2569.768802709"
Jul 23 01:06:06.786978 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:06.786944    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" path="/var/lib/kubelet/pods/6be27c27-63d6-489c-952b-bd6f1d8370d5/volumes"
Jul 23 01:06:09.118266 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.118230    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"]
Jul 23 01:06:09.118657 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.118576    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerName="istio-proxy"
Jul 23 01:06:09.118657 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.118588    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerName="istio-proxy"
Jul 23 01:06:09.118758 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.118676    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerName="istio-proxy"
Jul 23 01:06:09.121343 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.121324    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.124017 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.124000    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f05be8-openshift-default-dockercfg-cc65b\""
Jul 23 01:06:09.134053 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.134029    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"]
Jul 23 01:06:09.228923 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.228867    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-data\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.229117 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.228931    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-credential-socket\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.229117 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.228968    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-workload-certs\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.229117 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.229040    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfq55\" (UniqueName: \"kubernetes.io/projected/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-kube-api-access-jfq55\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.229117 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.229064    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istiod-ca-cert\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.229117 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.229091    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-podinfo\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.229401 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.229134    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-workload-socket\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.229401 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.229180    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-envoy\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.229401 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.229206    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-token\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.330103 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.330067    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfq55\" (UniqueName: \"kubernetes.io/projected/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-kube-api-access-jfq55\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.330299 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.330108    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istiod-ca-cert\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.330299 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.330137    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-podinfo\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.330299 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.330174    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-workload-socket\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.330299 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.330211    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-envoy\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.330299 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.330235    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-token\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.330299 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.330261    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-data\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.330299 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.330293    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-credential-socket\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.330701 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.330326    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-workload-certs\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.330701 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.330668    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-workload-socket\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.330810 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.330740    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-workload-certs\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.331019 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.330995    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istiod-ca-cert\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.331183 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.331040    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-credential-socket\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.331183 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.331056    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-data\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.332844 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.332815    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-envoy\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.333310 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.333292    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-podinfo\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.337996 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.337972    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-token\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.338150 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.338135    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfq55\" (UniqueName: \"kubernetes.io/projected/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-kube-api-access-jfq55\") pod \"e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") " pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.434728 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.434694    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:09.573391 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.573347    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"]
Jul 23 01:06:09.575469 ip-10-0-133-215 kubenswrapper[2561]: W0723 01:06:09.575439    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f08c7f7_aefa_4a92_8d49_3fb5e6de7d8b.slice/crio-9875b8e5c9e9e4a296ddee91a0851024e6d62f94f510cb19a2e6e5e0c7bff6db WatchSource:0}: Error finding container 9875b8e5c9e9e4a296ddee91a0851024e6d62f94f510cb19a2e6e5e0c7bff6db: Status 404 returned error can't find the container with id 9875b8e5c9e9e4a296ddee91a0851024e6d62f94f510cb19a2e6e5e0c7bff6db
Jul 23 01:06:09.577611 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.577576    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:06:09.577708 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.577658    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:06:09.577708 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.577698    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:06:09.965846 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.965806    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" event={"ID":"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b","Type":"ContainerStarted","Data":"7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2"}
Jul 23 01:06:09.966009 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.965869    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" event={"ID":"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b","Type":"ContainerStarted","Data":"9875b8e5c9e9e4a296ddee91a0851024e6d62f94f510cb19a2e6e5e0c7bff6db"}
Jul 23 01:06:09.992921 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:09.992865    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" podStartSLOduration=0.992847679 podStartE2EDuration="992.847679ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:06:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:06:09.991138273 +0000 UTC m=+2573.797549501" watchObservedRunningTime="2026-07-23 01:06:09.992847679 +0000 UTC m=+2573.799258911"
Jul 23 01:06:10.435194 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:10.435159    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:10.439984 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:10.439961    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:10.971061 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:10.971031    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:10.972043 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:10.972024    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:11.956408 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:11.956350    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:11.998770 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:11.998736    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"]
Jul 23 01:06:11.999166 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:11.999151    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerName="istio-proxy"
Jul 23 01:06:11.999212 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:11.999168    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerName="istio-proxy"
Jul 23 01:06:11.999262 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:11.999253    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6be27c27-63d6-489c-952b-bd6f1d8370d5" containerName="istio-proxy"
Jul 23 01:06:12.001455 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.001438    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:12.005465 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.005436    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f05be8-serving-cert\""
Jul 23 01:06:12.014616 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.014592    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"]
Jul 23 01:06:12.053680 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.053650    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prdxj\" (UniqueName: \"kubernetes.io/projected/3838042c-e615-43b1-af76-e2fd25e610d3-kube-api-access-prdxj\") pod \"maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:12.053870 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.053723    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3838042c-e615-43b1-af76-e2fd25e610d3-maas-api-tls\") pod \"maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:12.053870 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.053811    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3838042c-e615-43b1-af76-e2fd25e610d3-openshift-service-ca\") pod \"maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:12.154475 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.154427    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prdxj\" (UniqueName: \"kubernetes.io/projected/3838042c-e615-43b1-af76-e2fd25e610d3-kube-api-access-prdxj\") pod \"maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:12.154670 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.154516    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3838042c-e615-43b1-af76-e2fd25e610d3-maas-api-tls\") pod \"maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:12.154670 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.154564    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3838042c-e615-43b1-af76-e2fd25e610d3-openshift-service-ca\") pod \"maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:12.154670 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:06:12.154625    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-serving-cert: secret "maas-api-e2e-api-b-f05be8-serving-cert" not found
Jul 23 01:06:12.154895 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:06:12.154717    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3838042c-e615-43b1-af76-e2fd25e610d3-maas-api-tls podName:3838042c-e615-43b1-af76-e2fd25e610d3 nodeName:}" failed. No retries permitted until 2026-07-23 01:06:12.654693692 +0000 UTC m=+2576.461104913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3838042c-e615-43b1-af76-e2fd25e610d3-maas-api-tls") pod "maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5" (UID: "3838042c-e615-43b1-af76-e2fd25e610d3") : secret "maas-api-e2e-api-b-f05be8-serving-cert" not found
Jul 23 01:06:12.155412 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.155383    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3838042c-e615-43b1-af76-e2fd25e610d3-openshift-service-ca\") pod \"maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:12.166108 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.166071    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prdxj\" (UniqueName: \"kubernetes.io/projected/3838042c-e615-43b1-af76-e2fd25e610d3-kube-api-access-prdxj\") pod \"maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:12.659703 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.659667    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3838042c-e615-43b1-af76-e2fd25e610d3-maas-api-tls\") pod \"maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:12.662107 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.662087    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3838042c-e615-43b1-af76-e2fd25e610d3-maas-api-tls\") pod \"maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:12.914542 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:12.914451    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:13.061387 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:13.061335    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"]
Jul 23 01:06:13.063154 ip-10-0-133-215 kubenswrapper[2561]: W0723 01:06:13.063125    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3838042c_e615_43b1_af76_e2fd25e610d3.slice/crio-fa964691885ecb1077caa27a0b69fa61dc2755fced724f175ad775407a3243b6 WatchSource:0}: Error finding container fa964691885ecb1077caa27a0b69fa61dc2755fced724f175ad775407a3243b6: Status 404 returned error can't find the container with id fa964691885ecb1077caa27a0b69fa61dc2755fced724f175ad775407a3243b6
Jul 23 01:06:13.990991 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:13.990957    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5" event={"ID":"3838042c-e615-43b1-af76-e2fd25e610d3","Type":"ContainerStarted","Data":"3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e"}
Jul 23 01:06:13.991183 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:13.991016    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5" event={"ID":"3838042c-e615-43b1-af76-e2fd25e610d3","Type":"ContainerStarted","Data":"fa964691885ecb1077caa27a0b69fa61dc2755fced724f175ad775407a3243b6"}
Jul 23 01:06:13.991183 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:13.991048    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:14.007942 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:14.007885    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5" podStartSLOduration=2.607272875 podStartE2EDuration="3.007864091s" totalImagesPullingTime="400.591216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:06:14.007659082 +0000 UTC m=+2577.814070310" watchObservedRunningTime="2026-07-23 01:06:14.007864091 +0000 UTC m=+2577.814275325"
Jul 23 01:06:18.965510 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:18.965469    2561 patch_prober.go:28] interesting pod/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 23 01:06:18.965948 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:18.965533    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 23 01:06:20.004464 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:20.004428    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:06:21.027842 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:21.027805    2561 generic.go:356] "Generic (PLEG): container finished" podID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerID="9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827" exitCode=0
Jul 23 01:06:21.027842 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:21.027853    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" event={"ID":"814a5df0-5cd6-4f29-91c0-e88fa26b8490","Type":"ContainerDied","Data":"9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827"}
Jul 23 01:06:21.028341 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:21.028325    2561 scope.go:117] "RemoveContainer" containerID="9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827"
Jul 23 01:06:21.029419 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:21.029389    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:06:21.029534 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:21.029470    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:06:21.029599 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:21.029531    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:06:22.037866 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:22.037821    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" event={"ID":"814a5df0-5cd6-4f29-91c0-e88fa26b8490","Type":"ContainerStarted","Data":"7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b"}
Jul 23 01:06:22.890301 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:22.890266    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:22.890573 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:22.890404    2561 patch_prober.go:28] interesting pod/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 23 01:06:22.890573 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:22.890445    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 23 01:06:23.889655 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:23.889619    2561 patch_prober.go:28] interesting pod/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 23 01:06:23.890085 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:23.889673    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 23 01:06:24.890078 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:24.890041    2561 patch_prober.go:28] interesting pod/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 23 01:06:24.890504 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:24.890095    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 23 01:06:25.889787 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:25.889748    2561 patch_prober.go:28] interesting pod/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 23 01:06:25.889995 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:25.889821    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 23 01:06:26.894064 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:26.894026    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:26.894574 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:26.894488    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:26.895593 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:26.895570    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:29.082479 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:29.082445    2561 generic.go:356] "Generic (PLEG): container finished" podID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerID="7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2" exitCode=0
Jul 23 01:06:29.082934 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:29.082520    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" event={"ID":"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b","Type":"ContainerDied","Data":"7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2"}
Jul 23 01:06:29.082934 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:29.082901    2561 scope.go:117] "RemoveContainer" containerID="7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2"
Jul 23 01:06:29.083925 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:29.083899    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:06:29.083972 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:29.083963    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:06:29.084016 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:29.083996    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:06:30.089492 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:30.089461    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" event={"ID":"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b","Type":"ContainerStarted","Data":"b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0"}
Jul 23 01:06:30.435174 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:30.435132    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:30.435468 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:30.435440    2561 patch_prober.go:28] interesting pod/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 23 01:06:30.435564 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:30.435505    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 23 01:06:31.436431 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:31.435970    2561 patch_prober.go:28] interesting pod/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 23 01:06:31.436431 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:31.436034    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 23 01:06:32.435433 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:32.435387    2561 patch_prober.go:28] interesting pod/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 23 01:06:32.435665 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:32.435460    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 23 01:06:33.435840 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:33.435793    2561 patch_prober.go:28] interesting pod/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 23 01:06:33.436381 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:33.435871    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 23 01:06:34.439505 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:34.439477    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:34.439983 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:34.439857    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:34.440495 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:34.440473    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:06:40.270743 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:40.270535    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"]
Jul 23 01:06:40.271562 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:40.271527    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6" podUID="15f94540-dcc5-4c67-8584-553850c92f39" containerName="maas-api" containerID="cri-o://0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1" gracePeriod=30
Jul 23 01:06:40.538853 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:40.538790    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:40.730799 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:40.730766    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15f94540-dcc5-4c67-8584-553850c92f39-maas-api-tls\") pod \"15f94540-dcc5-4c67-8584-553850c92f39\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") "
Jul 23 01:06:40.730997 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:40.730827    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns5v6\" (UniqueName: \"kubernetes.io/projected/15f94540-dcc5-4c67-8584-553850c92f39-kube-api-access-ns5v6\") pod \"15f94540-dcc5-4c67-8584-553850c92f39\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") "
Jul 23 01:06:40.730997 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:40.730947    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f94540-dcc5-4c67-8584-553850c92f39-openshift-service-ca\") pod \"15f94540-dcc5-4c67-8584-553850c92f39\" (UID: \"15f94540-dcc5-4c67-8584-553850c92f39\") "
Jul 23 01:06:40.731381 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:40.731311    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15f94540-dcc5-4c67-8584-553850c92f39-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "15f94540-dcc5-4c67-8584-553850c92f39" (UID: "15f94540-dcc5-4c67-8584-553850c92f39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:06:40.732895 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:40.732863    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15f94540-dcc5-4c67-8584-553850c92f39-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "15f94540-dcc5-4c67-8584-553850c92f39" (UID: "15f94540-dcc5-4c67-8584-553850c92f39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 01:06:40.733035 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:40.733010    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15f94540-dcc5-4c67-8584-553850c92f39-kube-api-access-ns5v6" (OuterVolumeSpecName: "kube-api-access-ns5v6") pod "15f94540-dcc5-4c67-8584-553850c92f39" (UID: "15f94540-dcc5-4c67-8584-553850c92f39"). InnerVolumeSpecName "kube-api-access-ns5v6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:06:40.831742 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:40.831703    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns5v6\" (UniqueName: \"kubernetes.io/projected/15f94540-dcc5-4c67-8584-553850c92f39-kube-api-access-ns5v6\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:40.831742 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:40.831734    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f94540-dcc5-4c67-8584-553850c92f39-openshift-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:40.831742 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:40.831745    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15f94540-dcc5-4c67-8584-553850c92f39-maas-api-tls\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:41.144520 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:41.144429    2561 generic.go:356] "Generic (PLEG): container finished" podID="15f94540-dcc5-4c67-8584-553850c92f39" containerID="0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1" exitCode=0
Jul 23 01:06:41.144520 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:41.144472    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"
Jul 23 01:06:41.144520 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:41.144492    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6" event={"ID":"15f94540-dcc5-4c67-8584-553850c92f39","Type":"ContainerDied","Data":"0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1"}
Jul 23 01:06:41.144755 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:41.144546    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6" event={"ID":"15f94540-dcc5-4c67-8584-553850c92f39","Type":"ContainerDied","Data":"ba266b61b392a489c650671ad47f8b69a992a66d1996a2d611afb9f4c7ec7784"}
Jul 23 01:06:41.144755 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:41.144571    2561 scope.go:117] "RemoveContainer" containerID="0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1"
Jul 23 01:06:41.153048 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:41.153028    2561 scope.go:117] "RemoveContainer" containerID="0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1"
Jul 23 01:06:41.153308 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:06:41.153288    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1\": container with ID starting with 0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1 not found: ID does not exist" containerID="0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1"
Jul 23 01:06:41.153354 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:41.153317    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1"} err="failed to get container status \"0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1\": rpc error: code = NotFound desc = could not find container \"0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1\": container with ID starting with 0cb2b3643891182da6f3b94bbc8b6e8a020d5fd88e45d7b780f6da73bdc57ad1 not found: ID does not exist"
Jul 23 01:06:41.160605 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:41.160581    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"]
Jul 23 01:06:41.166605 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:41.166585    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c2ab89-574797b5c9-phpm6"]
Jul 23 01:06:42.786956 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:42.786905    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15f94540-dcc5-4c67-8584-553850c92f39" path="/var/lib/kubelet/pods/15f94540-dcc5-4c67-8584-553850c92f39/volumes"
Jul 23 01:06:50.872688 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:50.872652    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"]
Jul 23 01:06:50.873176 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:50.872868    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerName="istio-proxy" containerID="cri-o://7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b" gracePeriod=30
Jul 23 01:06:56.139725 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.139656    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:56.215578 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.215545    2561 generic.go:356] "Generic (PLEG): container finished" podID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerID="7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b" exitCode=0
Jul 23 01:06:56.215780 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.215586    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" event={"ID":"814a5df0-5cd6-4f29-91c0-e88fa26b8490","Type":"ContainerDied","Data":"7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b"}
Jul 23 01:06:56.215780 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.215595    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"
Jul 23 01:06:56.215780 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.215630    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4" event={"ID":"814a5df0-5cd6-4f29-91c0-e88fa26b8490","Type":"ContainerDied","Data":"85ebee85a667cd161d235be43c742f263adbdccddcea543d9d582079e736e17b"}
Jul 23 01:06:56.215780 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.215653    2561 scope.go:117] "RemoveContainer" containerID="7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b"
Jul 23 01:06:56.226326 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.226298    2561 scope.go:117] "RemoveContainer" containerID="9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827"
Jul 23 01:06:56.234327 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.234308    2561 scope.go:117] "RemoveContainer" containerID="7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b"
Jul 23 01:06:56.234598 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:06:56.234577    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b\": container with ID starting with 7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b not found: ID does not exist" containerID="7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b"
Jul 23 01:06:56.234659 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.234607    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b"} err="failed to get container status \"7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b\": rpc error: code = NotFound desc = could not find container \"7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b\": container with ID starting with 7f7704cd1c2d9514823c303b8e43529b5ce187a840a6bc2f36576665aeeb4d7b not found: ID does not exist"
Jul 23 01:06:56.234659 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.234626    2561 scope.go:117] "RemoveContainer" containerID="9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827"
Jul 23 01:06:56.234824 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:06:56.234803    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827\": container with ID starting with 9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827 not found: ID does not exist" containerID="9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827"
Jul 23 01:06:56.234862 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.234831    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827"} err="failed to get container status \"9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827\": rpc error: code = NotFound desc = could not find container \"9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827\": container with ID starting with 9ba3b73211c15e23a93c426daed41bdcbc97fc74c3696677abdc094b22558827 not found: ID does not exist"
Jul 23 01:06:56.264237 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264208    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-workload-certs\") pod \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") "
Jul 23 01:06:56.264356 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264244    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-token\") pod \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") "
Jul 23 01:06:56.264356 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264264    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-podinfo\") pod \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") "
Jul 23 01:06:56.264356 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264329    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-credential-socket\") pod \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") "
Jul 23 01:06:56.264522 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264374    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-workload-socket\") pod \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") "
Jul 23 01:06:56.264522 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264410    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mdht\" (UniqueName: \"kubernetes.io/projected/814a5df0-5cd6-4f29-91c0-e88fa26b8490-kube-api-access-2mdht\") pod \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") "
Jul 23 01:06:56.264522 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264441    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istiod-ca-cert\") pod \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") "
Jul 23 01:06:56.264522 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264510    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-envoy\") pod \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") "
Jul 23 01:06:56.264729 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264555    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-data\") pod \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\" (UID: \"814a5df0-5cd6-4f29-91c0-e88fa26b8490\") "
Jul 23 01:06:56.264729 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264671    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "814a5df0-5cd6-4f29-91c0-e88fa26b8490" (UID: "814a5df0-5cd6-4f29-91c0-e88fa26b8490"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:06:56.264839 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264757    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "814a5df0-5cd6-4f29-91c0-e88fa26b8490" (UID: "814a5df0-5cd6-4f29-91c0-e88fa26b8490"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:06:56.264839 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264770    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "814a5df0-5cd6-4f29-91c0-e88fa26b8490" (UID: "814a5df0-5cd6-4f29-91c0-e88fa26b8490"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:06:56.264938 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264855    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "814a5df0-5cd6-4f29-91c0-e88fa26b8490" (UID: "814a5df0-5cd6-4f29-91c0-e88fa26b8490"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:06:56.264938 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264886    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:56.264938 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264907    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:56.264938 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.264924    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:56.265527 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.265502    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-data" (OuterVolumeSpecName: "istio-data") pod "814a5df0-5cd6-4f29-91c0-e88fa26b8490" (UID: "814a5df0-5cd6-4f29-91c0-e88fa26b8490"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:06:56.266577 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.266555    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "814a5df0-5cd6-4f29-91c0-e88fa26b8490" (UID: "814a5df0-5cd6-4f29-91c0-e88fa26b8490"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:06:56.266835 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.266815    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "814a5df0-5cd6-4f29-91c0-e88fa26b8490" (UID: "814a5df0-5cd6-4f29-91c0-e88fa26b8490"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 01:06:56.266902 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.266884    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-token" (OuterVolumeSpecName: "istio-token") pod "814a5df0-5cd6-4f29-91c0-e88fa26b8490" (UID: "814a5df0-5cd6-4f29-91c0-e88fa26b8490"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:06:56.266986 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.266961    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/814a5df0-5cd6-4f29-91c0-e88fa26b8490-kube-api-access-2mdht" (OuterVolumeSpecName: "kube-api-access-2mdht") pod "814a5df0-5cd6-4f29-91c0-e88fa26b8490" (UID: "814a5df0-5cd6-4f29-91c0-e88fa26b8490"). InnerVolumeSpecName "kube-api-access-2mdht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:06:56.365576 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.365535    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:56.365576 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.365568    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:56.365576 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.365580    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:56.365829 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.365592    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:56.365829 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.365604    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mdht\" (UniqueName: \"kubernetes.io/projected/814a5df0-5cd6-4f29-91c0-e88fa26b8490-kube-api-access-2mdht\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:56.365829 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.365616    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/814a5df0-5cd6-4f29-91c0-e88fa26b8490-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:06:56.538377 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.538339    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"]
Jul 23 01:06:56.541889 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.541862    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c2ab89-openshift-default-667fc4d969-nk7g4"]
Jul 23 01:06:56.787070 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:06:56.786991    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" path="/var/lib/kubelet/pods/814a5df0-5cd6-4f29-91c0-e88fa26b8490/volumes"
Jul 23 01:07:06.677417 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:06.677380    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"]
Jul 23 01:07:06.678038 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:06.677601    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5" podUID="3838042c-e615-43b1-af76-e2fd25e610d3" containerName="maas-api" containerID="cri-o://3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e" gracePeriod=30
Jul 23 01:07:06.933547 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:06.933480    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:07:07.059828 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.059789    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3838042c-e615-43b1-af76-e2fd25e610d3-openshift-service-ca\") pod \"3838042c-e615-43b1-af76-e2fd25e610d3\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") "
Jul 23 01:07:07.060034 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.059842    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prdxj\" (UniqueName: \"kubernetes.io/projected/3838042c-e615-43b1-af76-e2fd25e610d3-kube-api-access-prdxj\") pod \"3838042c-e615-43b1-af76-e2fd25e610d3\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") "
Jul 23 01:07:07.060034 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.059868    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3838042c-e615-43b1-af76-e2fd25e610d3-maas-api-tls\") pod \"3838042c-e615-43b1-af76-e2fd25e610d3\" (UID: \"3838042c-e615-43b1-af76-e2fd25e610d3\") "
Jul 23 01:07:07.060246 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.060221    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3838042c-e615-43b1-af76-e2fd25e610d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3838042c-e615-43b1-af76-e2fd25e610d3" (UID: "3838042c-e615-43b1-af76-e2fd25e610d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:07:07.061961 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.061936    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3838042c-e615-43b1-af76-e2fd25e610d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3838042c-e615-43b1-af76-e2fd25e610d3" (UID: "3838042c-e615-43b1-af76-e2fd25e610d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 01:07:07.062105 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.062077    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3838042c-e615-43b1-af76-e2fd25e610d3-kube-api-access-prdxj" (OuterVolumeSpecName: "kube-api-access-prdxj") pod "3838042c-e615-43b1-af76-e2fd25e610d3" (UID: "3838042c-e615-43b1-af76-e2fd25e610d3"). InnerVolumeSpecName "kube-api-access-prdxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:07:07.160899 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.160852    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3838042c-e615-43b1-af76-e2fd25e610d3-openshift-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:07:07.160899 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.160891    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prdxj\" (UniqueName: \"kubernetes.io/projected/3838042c-e615-43b1-af76-e2fd25e610d3-kube-api-access-prdxj\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:07:07.160899 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.160904    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3838042c-e615-43b1-af76-e2fd25e610d3-maas-api-tls\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:07:07.269709 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.269669    2561 generic.go:356] "Generic (PLEG): container finished" podID="3838042c-e615-43b1-af76-e2fd25e610d3" containerID="3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e" exitCode=0
Jul 23 01:07:07.269899 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.269718    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"
Jul 23 01:07:07.269899 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.269718    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5" event={"ID":"3838042c-e615-43b1-af76-e2fd25e610d3","Type":"ContainerDied","Data":"3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e"}
Jul 23 01:07:07.269899 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.269773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5" event={"ID":"3838042c-e615-43b1-af76-e2fd25e610d3","Type":"ContainerDied","Data":"fa964691885ecb1077caa27a0b69fa61dc2755fced724f175ad775407a3243b6"}
Jul 23 01:07:07.269899 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.269805    2561 scope.go:117] "RemoveContainer" containerID="3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e"
Jul 23 01:07:07.283974 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.283950    2561 scope.go:117] "RemoveContainer" containerID="3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e"
Jul 23 01:07:07.284269 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:07:07.284249    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e\": container with ID starting with 3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e not found: ID does not exist" containerID="3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e"
Jul 23 01:07:07.284315 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.284276    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e"} err="failed to get container status \"3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e\": rpc error: code = NotFound desc = could not find container \"3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e\": container with ID starting with 3bdc663475a28932f89d86d6a9711ecef7b2690ba25defae8beb00127dce133e not found: ID does not exist"
Jul 23 01:07:07.296911 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.296885    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"]
Jul 23 01:07:07.301043 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:07.301018    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f05be8-86db7bfbb9-jlqw5"]
Jul 23 01:07:08.787518 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:08.787477    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3838042c-e615-43b1-af76-e2fd25e610d3" path="/var/lib/kubelet/pods/3838042c-e615-43b1-af76-e2fd25e610d3/volumes"
Jul 23 01:07:17.766500 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:17.766466    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"]
Jul 23 01:07:17.766962 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:17.766726    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerName="istio-proxy" containerID="cri-o://b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0" gracePeriod=30
Jul 23 01:07:18.628312 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.628274    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"]
Jul 23 01:07:18.628819 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.628799    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerName="istio-proxy"
Jul 23 01:07:18.628875 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.628826    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerName="istio-proxy"
Jul 23 01:07:18.628875 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.628847    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3838042c-e615-43b1-af76-e2fd25e610d3" containerName="maas-api"
Jul 23 01:07:18.628875 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.628856    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3838042c-e615-43b1-af76-e2fd25e610d3" containerName="maas-api"
Jul 23 01:07:18.628875 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.628866    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15f94540-dcc5-4c67-8584-553850c92f39" containerName="maas-api"
Jul 23 01:07:18.629037 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.628875    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="15f94540-dcc5-4c67-8584-553850c92f39" containerName="maas-api"
Jul 23 01:07:18.629037 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.628890    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerName="istio-proxy"
Jul 23 01:07:18.629037 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.628900    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerName="istio-proxy"
Jul 23 01:07:18.629037 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.628990    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="15f94540-dcc5-4c67-8584-553850c92f39" containerName="maas-api"
Jul 23 01:07:18.629037 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.629002    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerName="istio-proxy"
Jul 23 01:07:18.629037 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.629011    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3838042c-e615-43b1-af76-e2fd25e610d3" containerName="maas-api"
Jul 23 01:07:18.629037 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.629020    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="814a5df0-5cd6-4f29-91c0-e88fa26b8490" containerName="istio-proxy"
Jul 23 01:07:18.633723 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.633701    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.636292 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.636272    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-7ee5a5-openshift-default-dockercfg-xtlvq\""
Jul 23 01:07:18.652244 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.652207    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"]
Jul 23 01:07:18.760989 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.760949    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-workload-socket\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.760989 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.760994    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-token\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.761236 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.761019    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-data\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.761236 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.761101    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-podinfo\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.761236 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.761141    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-workload-certs\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.761236 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.761158    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istiod-ca-cert\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.761477 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.761257    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7gb4\" (UniqueName: \"kubernetes.io/projected/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-kube-api-access-h7gb4\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.761477 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.761336    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-envoy\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.761477 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.761403    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-credential-socket\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.862831 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.862795    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7gb4\" (UniqueName: \"kubernetes.io/projected/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-kube-api-access-h7gb4\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863325 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.862862    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-envoy\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863325 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.862884    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-credential-socket\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863325 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.862911    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-workload-socket\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863325 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.862942    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-token\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863325 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.862964    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-data\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863325 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.862996    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-podinfo\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863325 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.863038    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-workload-certs\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863325 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.863063    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istiod-ca-cert\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863760 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.863421    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-workload-socket\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863760 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.863498    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-credential-socket\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863760 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.863540    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-workload-certs\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863760 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.863555    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-data\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.863893 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.863782    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istiod-ca-cert\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.865268 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.865240    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-envoy\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.865641 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.865621    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-podinfo\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.884918 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.884851    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7gb4\" (UniqueName: \"kubernetes.io/projected/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-kube-api-access-h7gb4\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.887129 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.887110    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-token\") pod \"e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") " pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:18.955215 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:18.955188    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:19.094504 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:19.094480    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"]
Jul 23 01:07:19.095912 ip-10-0-133-215 kubenswrapper[2561]: W0723 01:07:19.095882    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bf1ccf8_ecb4_4188_9e93_2ec511bb55f4.slice/crio-e9a0d06a24b31388f4a524cd6349fe1a9059dd58abe7f0d36bdc87a3bef80655 WatchSource:0}: Error finding container e9a0d06a24b31388f4a524cd6349fe1a9059dd58abe7f0d36bdc87a3bef80655: Status 404 returned error can't find the container with id e9a0d06a24b31388f4a524cd6349fe1a9059dd58abe7f0d36bdc87a3bef80655
Jul 23 01:07:19.098032 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:19.098001    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:07:19.098079 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:19.098070    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:07:19.098115 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:19.098099    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:07:19.330211 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:19.330170    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7" event={"ID":"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4","Type":"ContainerStarted","Data":"43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa"}
Jul 23 01:07:19.330417 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:19.330238    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7" event={"ID":"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4","Type":"ContainerStarted","Data":"e9a0d06a24b31388f4a524cd6349fe1a9059dd58abe7f0d36bdc87a3bef80655"}
Jul 23 01:07:19.350351 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:19.350288    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7" podStartSLOduration=1.350267459 podStartE2EDuration="1.350267459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:07:19.349238916 +0000 UTC m=+2643.155650152" watchObservedRunningTime="2026-07-23 01:07:19.350267459 +0000 UTC m=+2643.156678692"
Jul 23 01:07:19.442687 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:19.442649    2561 patch_prober.go:28] interesting pod/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 23 01:07:19.442847 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:19.442703    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 01:07:19.955725 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:19.955686    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:20.960268 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:20.960236    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:21.340505 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:21.340423    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:21.341545 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:21.341525    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:23.026490 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.026429    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:07:23.203134 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.203104    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-podinfo\") pod \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") "
Jul 23 01:07:23.203134 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.203140    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-token\") pod \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") "
Jul 23 01:07:23.203416 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.203158    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-credential-socket\") pod \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") "
Jul 23 01:07:23.203416 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.203185    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfq55\" (UniqueName: \"kubernetes.io/projected/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-kube-api-access-jfq55\") pod \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") "
Jul 23 01:07:23.203416 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.203221    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-workload-certs\") pod \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") "
Jul 23 01:07:23.203589 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.203504    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" (UID: "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:07:23.203589 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.203519    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" (UID: "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:07:23.203589 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.203563    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istiod-ca-cert\") pod \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") "
Jul 23 01:07:23.203735 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.203629    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-data\") pod \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") "
Jul 23 01:07:23.203735 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.203693    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-workload-socket\") pod \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") "
Jul 23 01:07:23.203735 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.203727    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-envoy\") pod \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\" (UID: \"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b\") "
Jul 23 01:07:23.204026 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.204005    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" (UID: "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:07:23.204105 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.204046    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:07:23.204105 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.204050    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" (UID: "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:07:23.204105 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.204063    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:07:23.204633 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.204610    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-data" (OuterVolumeSpecName: "istio-data") pod "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" (UID: "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:07:23.205442 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.205418    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" (UID: "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 01:07:23.205827 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.205804    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" (UID: "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:07:23.205937 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.205918    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-kube-api-access-jfq55" (OuterVolumeSpecName: "kube-api-access-jfq55") pod "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" (UID: "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b"). InnerVolumeSpecName "kube-api-access-jfq55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:07:23.206098 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.206072    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-token" (OuterVolumeSpecName: "istio-token") pod "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" (UID: "2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:07:23.304850 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.304756    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:07:23.304850 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.304794    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:07:23.304850 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.304803    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:07:23.304850 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.304811    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:07:23.304850 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.304821    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfq55\" (UniqueName: \"kubernetes.io/projected/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-kube-api-access-jfq55\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:07:23.304850 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.304830    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:07:23.304850 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.304837    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:07:23.352082 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.352043    2561 generic.go:356] "Generic (PLEG): container finished" podID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerID="b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0" exitCode=0
Jul 23 01:07:23.352251 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.352097    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"
Jul 23 01:07:23.352251 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.352136    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" event={"ID":"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b","Type":"ContainerDied","Data":"b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0"}
Jul 23 01:07:23.352251 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.352185    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6" event={"ID":"2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b","Type":"ContainerDied","Data":"9875b8e5c9e9e4a296ddee91a0851024e6d62f94f510cb19a2e6e5e0c7bff6db"}
Jul 23 01:07:23.352251 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.352208    2561 scope.go:117] "RemoveContainer" containerID="b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0"
Jul 23 01:07:23.361536 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.361517    2561 scope.go:117] "RemoveContainer" containerID="7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2"
Jul 23 01:07:23.369549 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.369531    2561 scope.go:117] "RemoveContainer" containerID="b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0"
Jul 23 01:07:23.369803 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:07:23.369784    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0\": container with ID starting with b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0 not found: ID does not exist" containerID="b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0"
Jul 23 01:07:23.369851 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.369812    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0"} err="failed to get container status \"b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0\": rpc error: code = NotFound desc = could not find container \"b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0\": container with ID starting with b59d13dbaf8b789f154b90e03530f4f533164392db64c6a89245d2340c533ac0 not found: ID does not exist"
Jul 23 01:07:23.369851 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.369829    2561 scope.go:117] "RemoveContainer" containerID="7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2"
Jul 23 01:07:23.370085 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:07:23.370062    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2\": container with ID starting with 7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2 not found: ID does not exist" containerID="7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2"
Jul 23 01:07:23.370130 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.370094    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2"} err="failed to get container status \"7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2\": rpc error: code = NotFound desc = could not find container \"7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2\": container with ID starting with 7b849ef448b048e1a2af131796ebc0a35b80a2565e910bda424967368abebec2 not found: ID does not exist"
Jul 23 01:07:23.380611 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.380558    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"]
Jul 23 01:07:23.382539 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:23.382516    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f05be8-openshift-default-656c546d7c-kldz6"]
Jul 23 01:07:24.787451 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:24.787417    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" path="/var/lib/kubelet/pods/2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b/volumes"
Jul 23 01:07:26.454309 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.454276    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"]
Jul 23 01:07:26.454764 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.454685    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerName="istio-proxy"
Jul 23 01:07:26.454764 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.454701    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerName="istio-proxy"
Jul 23 01:07:26.454764 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.454712    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerName="istio-proxy"
Jul 23 01:07:26.454764 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.454717    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerName="istio-proxy"
Jul 23 01:07:26.454998 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.454785    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerName="istio-proxy"
Jul 23 01:07:26.457800 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.457778    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.461538 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.461520    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-d6869a-openshift-default-dockercfg-9m5gk\""
Jul 23 01:07:26.470091 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.470067    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"]
Jul 23 01:07:26.633143 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.633099    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-podinfo\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.633322 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.633158    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-token\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.633322 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.633209    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmb2s\" (UniqueName: \"kubernetes.io/projected/9fba2c9a-e730-4f44-8186-5b623e65a745-kube-api-access-nmb2s\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.633322 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.633246    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-workload-socket\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.633322 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.633270    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-workload-certs\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.633322 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.633294    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fba2c9a-e730-4f44-8186-5b623e65a745-istiod-ca-cert\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.633553 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.633378    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-credential-socket\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.633553 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.633425    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-envoy\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.633553 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.633455    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-data\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.734399 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.734301    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-podinfo\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.734399 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.734341    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-token\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.734399 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.734389    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmb2s\" (UniqueName: \"kubernetes.io/projected/9fba2c9a-e730-4f44-8186-5b623e65a745-kube-api-access-nmb2s\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.734682 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.734412    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-workload-socket\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.734682 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.734429    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-workload-certs\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.734682 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.734446    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fba2c9a-e730-4f44-8186-5b623e65a745-istiod-ca-cert\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.734682 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.734518    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-credential-socket\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.734682 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.734579    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-envoy\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.734682 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.734605    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-data\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.735027 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.734916    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-workload-socket\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.735027 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.734954    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-credential-socket\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.735402 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.735322    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-workload-certs\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.735402 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.735323    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-data\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.735649 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.735623    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fba2c9a-e730-4f44-8186-5b623e65a745-istiod-ca-cert\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.737011 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.736982    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-envoy\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.737121 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.737095    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-podinfo\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.747590 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.747529    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-token\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.754959 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.754935    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmb2s\" (UniqueName: \"kubernetes.io/projected/9fba2c9a-e730-4f44-8186-5b623e65a745-kube-api-access-nmb2s\") pod \"e2e-shared-b-d6869a-openshift-default-998997579-cfll5\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") " pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.768618 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.768595    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:26.906386 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.906347    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"]
Jul 23 01:07:26.907694 ip-10-0-133-215 kubenswrapper[2561]: W0723 01:07:26.907659    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fba2c9a_e730_4f44_8186_5b623e65a745.slice/crio-92e952b9d634a2c6204b5264e752f1251f85f268d04a93b0113492f774a9f0ee WatchSource:0}: Error finding container 92e952b9d634a2c6204b5264e752f1251f85f268d04a93b0113492f774a9f0ee: Status 404 returned error can't find the container with id 92e952b9d634a2c6204b5264e752f1251f85f268d04a93b0113492f774a9f0ee
Jul 23 01:07:26.909870 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.909841    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:07:26.909921 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.909904    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:07:26.909966 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:26.909933    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:07:27.376569 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:27.376536    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5" event={"ID":"9fba2c9a-e730-4f44-8186-5b623e65a745","Type":"ContainerStarted","Data":"65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965"}
Jul 23 01:07:27.376775 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:27.376594    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5" event={"ID":"9fba2c9a-e730-4f44-8186-5b623e65a745","Type":"ContainerStarted","Data":"92e952b9d634a2c6204b5264e752f1251f85f268d04a93b0113492f774a9f0ee"}
Jul 23 01:07:27.415459 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:27.415404    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5" podStartSLOduration=1.415387792 podStartE2EDuration="1.415387792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:07:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:07:27.413633239 +0000 UTC m=+2651.220044483" watchObservedRunningTime="2026-07-23 01:07:27.415387792 +0000 UTC m=+2651.221799025"
Jul 23 01:07:27.769441 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:27.769404    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:27.774176 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:27.774152    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:28.381758 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:28.381722    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:28.382992 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:28.382972    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:29.454321 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.454273    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"]
Jul 23 01:07:29.454991 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.454967    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f08c7f7-aefa-4a92-8d49-3fb5e6de7d8b" containerName="istio-proxy"
Jul 23 01:07:29.458341 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.458319    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:29.461592 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.461570    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-d6869a-serving-cert\""
Jul 23 01:07:29.467354 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.467328    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00748046-7284-4250-98d2-e8ac0d9a5148-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb\" (UID: \"00748046-7284-4250-98d2-e8ac0d9a5148\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:29.467571 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.467548    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s94rj\" (UniqueName: \"kubernetes.io/projected/00748046-7284-4250-98d2-e8ac0d9a5148-kube-api-access-s94rj\") pod \"maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb\" (UID: \"00748046-7284-4250-98d2-e8ac0d9a5148\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:29.467789 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.467770    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00748046-7284-4250-98d2-e8ac0d9a5148-maas-api-tls\") pod \"maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb\" (UID: \"00748046-7284-4250-98d2-e8ac0d9a5148\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:29.471211 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.471156    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"]
Jul 23 01:07:29.568855 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.568814    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00748046-7284-4250-98d2-e8ac0d9a5148-maas-api-tls\") pod \"maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb\" (UID: \"00748046-7284-4250-98d2-e8ac0d9a5148\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:29.569011 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.568887    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00748046-7284-4250-98d2-e8ac0d9a5148-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb\" (UID: \"00748046-7284-4250-98d2-e8ac0d9a5148\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:29.569011 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.568917    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s94rj\" (UniqueName: \"kubernetes.io/projected/00748046-7284-4250-98d2-e8ac0d9a5148-kube-api-access-s94rj\") pod \"maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb\" (UID: \"00748046-7284-4250-98d2-e8ac0d9a5148\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:29.569655 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.569633    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00748046-7284-4250-98d2-e8ac0d9a5148-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb\" (UID: \"00748046-7284-4250-98d2-e8ac0d9a5148\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:29.571340 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.571314    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00748046-7284-4250-98d2-e8ac0d9a5148-maas-api-tls\") pod \"maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb\" (UID: \"00748046-7284-4250-98d2-e8ac0d9a5148\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:29.579125 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.579103    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s94rj\" (UniqueName: \"kubernetes.io/projected/00748046-7284-4250-98d2-e8ac0d9a5148-kube-api-access-s94rj\") pod \"maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb\" (UID: \"00748046-7284-4250-98d2-e8ac0d9a5148\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:29.773126 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.773038    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:29.926339 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:29.926288    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"]
Jul 23 01:07:30.394209 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:30.394169    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb" event={"ID":"00748046-7284-4250-98d2-e8ac0d9a5148","Type":"ContainerStarted","Data":"1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7"}
Jul 23 01:07:30.394315 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:30.394232    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb" event={"ID":"00748046-7284-4250-98d2-e8ac0d9a5148","Type":"ContainerStarted","Data":"281d4de929a699a5c05052975601a3e271a8c490c80ded778dc7ef8ce4ddee28"}
Jul 23 01:07:30.394408 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:30.394390    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:30.415538 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:30.415486    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb" podStartSLOduration=1.063318013 podStartE2EDuration="1.415470557s" totalImagesPullingTime="352.152544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:07:30.413060195 +0000 UTC m=+2654.219471427" watchObservedRunningTime="2026-07-23 01:07:30.415470557 +0000 UTC m=+2654.221881789"
Jul 23 01:07:35.419317 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:35.419281    2561 generic.go:356] "Generic (PLEG): container finished" podID="0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" containerID="43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa" exitCode=0
Jul 23 01:07:35.419882 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:35.419353    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7" event={"ID":"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4","Type":"ContainerDied","Data":"43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa"}
Jul 23 01:07:35.419882 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:35.419853    2561 scope.go:117] "RemoveContainer" containerID="43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa"
Jul 23 01:07:35.420878 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:35.420849    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:07:35.420988 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:35.420934    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:07:35.420988 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:35.420980    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:07:36.340555 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:36.340514    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:36.404979 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:36.404949    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:07:36.427113 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:36.427075    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7" event={"ID":"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4","Type":"ContainerStarted","Data":"3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3"}
Jul 23 01:07:36.957085 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:36.957041    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:36.957340 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:36.957195    2561 patch_prober.go:28] interesting pod/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 23 01:07:36.957340 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:36.957240    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7" podUID="0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 23 01:07:37.955801 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:37.955764    2561 patch_prober.go:28] interesting pod/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 23 01:07:37.957123 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:37.956728    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7" podUID="0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 23 01:07:38.955754 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:38.955708    2561 patch_prober.go:28] interesting pod/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 23 01:07:38.955971 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:38.955793    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7" podUID="0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 23 01:07:39.957847 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:39.957343    2561 patch_prober.go:28] interesting pod/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 23 01:07:39.957847 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:39.957446    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7" podUID="0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 23 01:07:40.960330 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:40.960300    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:40.960735 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:40.960712    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:40.961400 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:40.961380    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:07:42.489536 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:42.489502    2561 generic.go:356] "Generic (PLEG): container finished" podID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerID="65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965" exitCode=0
Jul 23 01:07:42.489988 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:42.489593    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5" event={"ID":"9fba2c9a-e730-4f44-8186-5b623e65a745","Type":"ContainerDied","Data":"65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965"}
Jul 23 01:07:42.490126 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:42.490108    2561 scope.go:117] "RemoveContainer" containerID="65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965"
Jul 23 01:07:42.491090 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:42.491063    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:07:42.491191 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:42.491143    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:07:42.491273 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:42.491191    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 01:07:43.382009 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:43.381976    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:43.496075 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:43.496043    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5" event={"ID":"9fba2c9a-e730-4f44-8186-5b623e65a745","Type":"ContainerStarted","Data":"0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd"}
Jul 23 01:07:43.769461 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:43.769403    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:43.770347 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:43.770324    2561 patch_prober.go:28] interesting pod/e2e-shared-b-d6869a-openshift-default-998997579-cfll5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 23 01:07:43.770438 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:43.770395    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5" podUID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 23 01:07:44.769295 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:44.769262    2561 patch_prober.go:28] interesting pod/e2e-shared-b-d6869a-openshift-default-998997579-cfll5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 23 01:07:44.769832 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:44.769803    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5" podUID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 23 01:07:45.769698 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:45.769657    2561 patch_prober.go:28] interesting pod/e2e-shared-b-d6869a-openshift-default-998997579-cfll5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 23 01:07:45.770148 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:45.770122    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5" podUID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 23 01:07:46.769154 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:46.769119    2561 patch_prober.go:28] interesting pod/e2e-shared-b-d6869a-openshift-default-998997579-cfll5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 23 01:07:46.769347 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:46.769183    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5" podUID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 23 01:07:47.773136 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:47.773107    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:47.773564 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:47.773376    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:07:47.774271 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:07:47.774254    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:08:16.859692 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:08:16.859615    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 01:08:16.863621 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:08:16.863601    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 01:13:16.887180 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:16.887146    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 01:13:16.891419 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:16.891399    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g9g57_0c1097a5-8fea-43ca-be95-5f24ef1ebda4/ovn-acl-logging/0.log"
Jul 23 01:13:30.281667 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:30.281625    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"]
Jul 23 01:13:30.282173 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:30.281894    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7" podUID="0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" containerName="istio-proxy" containerID="cri-o://3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3" gracePeriod=30
Jul 23 01:13:35.531086 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.531058    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:13:35.579152 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579059    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-credential-socket\") pod \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") "
Jul 23 01:13:35.579152 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579114    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7gb4\" (UniqueName: \"kubernetes.io/projected/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-kube-api-access-h7gb4\") pod \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") "
Jul 23 01:13:35.579152 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579148    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-workload-certs\") pod \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") "
Jul 23 01:13:35.579453 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579179    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istiod-ca-cert\") pod \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") "
Jul 23 01:13:35.579453 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579201    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-podinfo\") pod \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") "
Jul 23 01:13:35.579453 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579218    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-token\") pod \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") "
Jul 23 01:13:35.579453 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579406    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" (UID: "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:13:35.579663 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579559    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" (UID: "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:13:35.579663 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579592    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-data\") pod \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") "
Jul 23 01:13:35.579663 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579648    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-workload-socket\") pod \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") "
Jul 23 01:13:35.579663 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579648    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" (UID: "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:13:35.579852 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579675    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-envoy\") pod \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\" (UID: \"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4\") "
Jul 23 01:13:35.579943 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579916    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" (UID: "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:13:35.580018 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579971    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:35.580018 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.579991    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:35.580018 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.580005    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:35.580648 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.580618    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-data" (OuterVolumeSpecName: "istio-data") pod "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" (UID: "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:13:35.581578 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.581550    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-token" (OuterVolumeSpecName: "istio-token") pod "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" (UID: "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:13:35.581857 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.581840    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" (UID: "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:13:35.581956 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.581935    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-kube-api-access-h7gb4" (OuterVolumeSpecName: "kube-api-access-h7gb4") pod "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" (UID: "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4"). InnerVolumeSpecName "kube-api-access-h7gb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:13:35.581997 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.581937    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" (UID: "0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 01:13:35.681181 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.681151    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:35.681181 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.681178    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:35.681181 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.681189    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:35.681423 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.681198    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7gb4\" (UniqueName: \"kubernetes.io/projected/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-kube-api-access-h7gb4\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:35.681423 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.681209    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:35.681423 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:35.681217    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:36.140041 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.140003    2561 generic.go:356] "Generic (PLEG): container finished" podID="0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" containerID="3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3" exitCode=0
Jul 23 01:13:36.140217 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.140047    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"
Jul 23 01:13:36.140217 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.140091    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7" event={"ID":"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4","Type":"ContainerDied","Data":"3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3"}
Jul 23 01:13:36.140217 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.140150    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7" event={"ID":"0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4","Type":"ContainerDied","Data":"e9a0d06a24b31388f4a524cd6349fe1a9059dd58abe7f0d36bdc87a3bef80655"}
Jul 23 01:13:36.140217 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.140173    2561 scope.go:117] "RemoveContainer" containerID="3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3"
Jul 23 01:13:36.149270 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.149250    2561 scope.go:117] "RemoveContainer" containerID="43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa"
Jul 23 01:13:36.157258 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.157236    2561 scope.go:117] "RemoveContainer" containerID="3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3"
Jul 23 01:13:36.157555 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:13:36.157536    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3\": container with ID starting with 3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3 not found: ID does not exist" containerID="3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3"
Jul 23 01:13:36.157621 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.157563    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3"} err="failed to get container status \"3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3\": rpc error: code = NotFound desc = could not find container \"3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3\": container with ID starting with 3bc4c303a3ed2acbdfa1f858100a17533b2b52e38cc4034630bd52d0159eb1d3 not found: ID does not exist"
Jul 23 01:13:36.157621 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.157582    2561 scope.go:117] "RemoveContainer" containerID="43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa"
Jul 23 01:13:36.157798 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:13:36.157782    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa\": container with ID starting with 43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa not found: ID does not exist" containerID="43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa"
Jul 23 01:13:36.157839 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.157803    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa"} err="failed to get container status \"43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa\": rpc error: code = NotFound desc = could not find container \"43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa\": container with ID starting with 43c11abaa9548264daffa4dbcaecedc0a4a5772f0c523efa20718cba24bea0fa not found: ID does not exist"
Jul 23 01:13:36.161349 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.161326    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"]
Jul 23 01:13:36.166689 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.166669    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-7ee5a5-openshift-default-d6f44467f-vpgc7"]
Jul 23 01:13:36.787845 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:36.787810    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" path="/var/lib/kubelet/pods/0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4/volumes"
Jul 23 01:13:41.130271 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:41.130232    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"]
Jul 23 01:13:41.130811 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:41.130506    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb" podUID="00748046-7284-4250-98d2-e8ac0d9a5148" containerName="maas-api" containerID="cri-o://1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7" gracePeriod=30
Jul 23 01:13:41.385067 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:41.385001    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:13:41.428174 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:41.428146    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s94rj\" (UniqueName: \"kubernetes.io/projected/00748046-7284-4250-98d2-e8ac0d9a5148-kube-api-access-s94rj\") pod \"00748046-7284-4250-98d2-e8ac0d9a5148\" (UID: \"00748046-7284-4250-98d2-e8ac0d9a5148\") "
Jul 23 01:13:41.428396 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:41.428214    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00748046-7284-4250-98d2-e8ac0d9a5148-maas-api-tls\") pod \"00748046-7284-4250-98d2-e8ac0d9a5148\" (UID: \"00748046-7284-4250-98d2-e8ac0d9a5148\") "
Jul 23 01:13:41.428396 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:41.428267    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00748046-7284-4250-98d2-e8ac0d9a5148-openshift-service-ca\") pod \"00748046-7284-4250-98d2-e8ac0d9a5148\" (UID: \"00748046-7284-4250-98d2-e8ac0d9a5148\") "
Jul 23 01:13:41.428737 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:41.428706    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00748046-7284-4250-98d2-e8ac0d9a5148-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "00748046-7284-4250-98d2-e8ac0d9a5148" (UID: "00748046-7284-4250-98d2-e8ac0d9a5148"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:13:41.431254 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:41.431206    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00748046-7284-4250-98d2-e8ac0d9a5148-kube-api-access-s94rj" (OuterVolumeSpecName: "kube-api-access-s94rj") pod "00748046-7284-4250-98d2-e8ac0d9a5148" (UID: "00748046-7284-4250-98d2-e8ac0d9a5148"). InnerVolumeSpecName "kube-api-access-s94rj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:13:41.431620 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:41.431590    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00748046-7284-4250-98d2-e8ac0d9a5148-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "00748046-7284-4250-98d2-e8ac0d9a5148" (UID: "00748046-7284-4250-98d2-e8ac0d9a5148"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 01:13:41.529343 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:41.529174    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s94rj\" (UniqueName: \"kubernetes.io/projected/00748046-7284-4250-98d2-e8ac0d9a5148-kube-api-access-s94rj\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:41.529343 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:41.529239    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00748046-7284-4250-98d2-e8ac0d9a5148-maas-api-tls\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:41.529343 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:41.529263    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00748046-7284-4250-98d2-e8ac0d9a5148-openshift-service-ca\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:42.173057 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:42.173024    2561 generic.go:356] "Generic (PLEG): container finished" podID="00748046-7284-4250-98d2-e8ac0d9a5148" containerID="1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7" exitCode=0
Jul 23 01:13:42.173571 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:42.173085    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"
Jul 23 01:13:42.173571 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:42.173113    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb" event={"ID":"00748046-7284-4250-98d2-e8ac0d9a5148","Type":"ContainerDied","Data":"1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7"}
Jul 23 01:13:42.173571 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:42.173182    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb" event={"ID":"00748046-7284-4250-98d2-e8ac0d9a5148","Type":"ContainerDied","Data":"281d4de929a699a5c05052975601a3e271a8c490c80ded778dc7ef8ce4ddee28"}
Jul 23 01:13:42.173571 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:42.173219    2561 scope.go:117] "RemoveContainer" containerID="1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7"
Jul 23 01:13:42.182049 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:42.182033    2561 scope.go:117] "RemoveContainer" containerID="1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7"
Jul 23 01:13:42.182274 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:13:42.182255    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7\": container with ID starting with 1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7 not found: ID does not exist" containerID="1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7"
Jul 23 01:13:42.182322 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:42.182282    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7"} err="failed to get container status \"1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7\": rpc error: code = NotFound desc = could not find container \"1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7\": container with ID starting with 1725b286826aa68abf780308987b22565ef6c0311429ce86c743bde9ddde5cc7 not found: ID does not exist"
Jul 23 01:13:42.193959 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:42.193933    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"]
Jul 23 01:13:42.197639 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:42.197619    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d6869a-66dc5f96c4-46sbb"]
Jul 23 01:13:42.787910 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:42.787879    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00748046-7284-4250-98d2-e8ac0d9a5148" path="/var/lib/kubelet/pods/00748046-7284-4250-98d2-e8ac0d9a5148/volumes"
Jul 23 01:13:52.312214 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:52.312180    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"]
Jul 23 01:13:52.312803 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:52.312419    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5" podUID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerName="istio-proxy" containerID="cri-o://0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd" gracePeriod=30
Jul 23 01:13:57.571426 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.571394    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:13:57.672597 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.672515    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-credential-socket\") pod \"9fba2c9a-e730-4f44-8186-5b623e65a745\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") "
Jul 23 01:13:57.672597 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.672559    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-envoy\") pod \"9fba2c9a-e730-4f44-8186-5b623e65a745\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") "
Jul 23 01:13:57.672597 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.672597    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmb2s\" (UniqueName: \"kubernetes.io/projected/9fba2c9a-e730-4f44-8186-5b623e65a745-kube-api-access-nmb2s\") pod \"9fba2c9a-e730-4f44-8186-5b623e65a745\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") "
Jul 23 01:13:57.672875 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.672614    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-workload-certs\") pod \"9fba2c9a-e730-4f44-8186-5b623e65a745\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") "
Jul 23 01:13:57.672875 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.672636    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fba2c9a-e730-4f44-8186-5b623e65a745-istiod-ca-cert\") pod \"9fba2c9a-e730-4f44-8186-5b623e65a745\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") "
Jul 23 01:13:57.672875 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.672657    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-podinfo\") pod \"9fba2c9a-e730-4f44-8186-5b623e65a745\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") "
Jul 23 01:13:57.672875 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.672692    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-data\") pod \"9fba2c9a-e730-4f44-8186-5b623e65a745\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") "
Jul 23 01:13:57.672875 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.672724    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-workload-socket\") pod \"9fba2c9a-e730-4f44-8186-5b623e65a745\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") "
Jul 23 01:13:57.672875 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.672790    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9fba2c9a-e730-4f44-8186-5b623e65a745" (UID: "9fba2c9a-e730-4f44-8186-5b623e65a745"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:13:57.673203 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.672897    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-token\") pod \"9fba2c9a-e730-4f44-8186-5b623e65a745\" (UID: \"9fba2c9a-e730-4f44-8186-5b623e65a745\") "
Jul 23 01:13:57.673203 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.673109    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9fba2c9a-e730-4f44-8186-5b623e65a745" (UID: "9fba2c9a-e730-4f44-8186-5b623e65a745"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:13:57.673203 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.673126    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fba2c9a-e730-4f44-8186-5b623e65a745-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9fba2c9a-e730-4f44-8186-5b623e65a745" (UID: "9fba2c9a-e730-4f44-8186-5b623e65a745"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 01:13:57.673203 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.673193    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-credential-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:57.673464 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.673211    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-workload-certs\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:57.673464 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.673336    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9fba2c9a-e730-4f44-8186-5b623e65a745" (UID: "9fba2c9a-e730-4f44-8186-5b623e65a745"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:13:57.673655 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.673623    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-data" (OuterVolumeSpecName: "istio-data") pod "9fba2c9a-e730-4f44-8186-5b623e65a745" (UID: "9fba2c9a-e730-4f44-8186-5b623e65a745"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:13:57.674941 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.674906    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fba2c9a-e730-4f44-8186-5b623e65a745-kube-api-access-nmb2s" (OuterVolumeSpecName: "kube-api-access-nmb2s") pod "9fba2c9a-e730-4f44-8186-5b623e65a745" (UID: "9fba2c9a-e730-4f44-8186-5b623e65a745"). InnerVolumeSpecName "kube-api-access-nmb2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:13:57.674941 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.674902    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9fba2c9a-e730-4f44-8186-5b623e65a745" (UID: "9fba2c9a-e730-4f44-8186-5b623e65a745"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 01:13:57.675112 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.675018    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9fba2c9a-e730-4f44-8186-5b623e65a745" (UID: "9fba2c9a-e730-4f44-8186-5b623e65a745"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 01:13:57.675112 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.675028    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-token" (OuterVolumeSpecName: "istio-token") pod "9fba2c9a-e730-4f44-8186-5b623e65a745" (UID: "9fba2c9a-e730-4f44-8186-5b623e65a745"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 01:13:57.774254 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.774215    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-podinfo\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:57.774254 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.774246    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-data\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:57.774254 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.774256    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-workload-socket\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:57.774254 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.774264    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-token\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:57.774556 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.774274    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fba2c9a-e730-4f44-8186-5b623e65a745-istio-envoy\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:57.774556 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.774284    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmb2s\" (UniqueName: \"kubernetes.io/projected/9fba2c9a-e730-4f44-8186-5b623e65a745-kube-api-access-nmb2s\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:57.774556 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:57.774292    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fba2c9a-e730-4f44-8186-5b623e65a745-istiod-ca-cert\") on node \"ip-10-0-133-215.ec2.internal\" DevicePath \"\""
Jul 23 01:13:58.246832 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.246800    2561 generic.go:356] "Generic (PLEG): container finished" podID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerID="0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd" exitCode=0
Jul 23 01:13:58.247014 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.246847    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"
Jul 23 01:13:58.247014 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.246880    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5" event={"ID":"9fba2c9a-e730-4f44-8186-5b623e65a745","Type":"ContainerDied","Data":"0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd"}
Jul 23 01:13:58.247014 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.246939    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5" event={"ID":"9fba2c9a-e730-4f44-8186-5b623e65a745","Type":"ContainerDied","Data":"92e952b9d634a2c6204b5264e752f1251f85f268d04a93b0113492f774a9f0ee"}
Jul 23 01:13:58.247014 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.246967    2561 scope.go:117] "RemoveContainer" containerID="0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd"
Jul 23 01:13:58.256309 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.256293    2561 scope.go:117] "RemoveContainer" containerID="65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965"
Jul 23 01:13:58.263748 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.263731    2561 scope.go:117] "RemoveContainer" containerID="0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd"
Jul 23 01:13:58.264011 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:13:58.263988    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd\": container with ID starting with 0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd not found: ID does not exist" containerID="0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd"
Jul 23 01:13:58.264056 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.264019    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd"} err="failed to get container status \"0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd\": rpc error: code = NotFound desc = could not find container \"0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd\": container with ID starting with 0cf551076df341c3f076c8f1f9cf6bc8f6cc38a20262f5bc988b5aada786e5cd not found: ID does not exist"
Jul 23 01:13:58.264056 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.264036    2561 scope.go:117] "RemoveContainer" containerID="65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965"
Jul 23 01:13:58.264258 ip-10-0-133-215 kubenswrapper[2561]: E0723 01:13:58.264239    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965\": container with ID starting with 65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965 not found: ID does not exist" containerID="65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965"
Jul 23 01:13:58.264306 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.264265    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965"} err="failed to get container status \"65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965\": rpc error: code = NotFound desc = could not find container \"65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965\": container with ID starting with 65c8dfa6cb91b06f18e5265986791f9d4f3c553de8cf4121f419024b13b4a965 not found: ID does not exist"
Jul 23 01:13:58.269203 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.269181    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"]
Jul 23 01:13:58.273988 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.273968    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-d6869a-openshift-default-998997579-cfll5"]
Jul 23 01:13:58.583155 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.583080    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kh6dp_816a95a0-4fa8-48ff-9486-45ac658ee1f2/manager/0.log"
Jul 23 01:13:58.712467 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.712440    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rnpwl_c5f8583e-370b-4998-a270-4e09f3bb2722/manager/0.log"
Jul 23 01:13:58.787381 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.787344    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fba2c9a-e730-4f44-8186-5b623e65a745" path="/var/lib/kubelet/pods/9fba2c9a-e730-4f44-8186-5b623e65a745/volumes"
Jul 23 01:13:58.829413 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.829384    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-795c797fc6-c5v62_b94b6c9a-67da-4989-a5d3-131b1b3352e3/manager/0.log"
Jul 23 01:13:58.946137 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:58.946109    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qqlgk_bc9b2386-9099-45b6-bc22-bbe13ae267c8/server/0.log"
Jul 23 01:13:59.346228 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:59.346145    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-695645577-bn7qh_16466a8f-10c0-4a8e-a961-221d79f2b668/copy-manifests/0.log"
Jul 23 01:13:59.351884 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:59.351857    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-695645577-bn7qh_16466a8f-10c0-4a8e-a961-221d79f2b668/manager/0.log"
Jul 23 01:13:59.852264 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:59.852228    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6d6b88c5c-mrx24_eee8f05c-892d-4bfd-8baf-604460a5a407/maas-api/0.log"
Jul 23 01:13:59.992832 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:13:59.992802    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29746125-2mtj8_32a66aea-b659-4149-9457-7f18e46602a8/cleanup/0.log"
Jul 23 01:14:00.235147 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:00.235124    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-2cb44_792026ff-c2fc-46f0-abd9-ead17efcde54/postgres/0.log"
Jul 23 01:14:01.850419 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:01.850385    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-txt2m_0fa0eb2d-2929-44ba-8cb3-d5b0bf360756/manager/0.log"
Jul 23 01:14:02.259191 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:02.259142    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bfx9c_6e7e0f2c-3558-4054-b460-2da581eb40fd/manager/0.log"
Jul 23 01:14:03.006044 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:03.006016    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-g2w67_760e166b-3e6c-47c6-b2f6-3ab7b4180f53/discovery/0.log"
Jul 23 01:14:03.124459 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:03.124428    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-678df7485d-fgqcj_360bf7a3-397c-47be-b280-da4c534f26dd/kube-auth-proxy/0.log"
Jul 23 01:14:03.499230 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:03.499200    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zzxjj_6ded7055-6424-4dd9-8968-b6f3e7ecad22/payload-pre-processing/0.log"
Jul 23 01:14:03.620943 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:03.620912    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-2z2g7_5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904/payload-processing/0.log"
Jul 23 01:14:04.209819 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:04.209792    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-9xx4m_0a7740bf-117c-4293-9d1f-292fa3e40d94/main/0.log"
Jul 23 01:14:13.951517 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:13.951438    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-l8zzv_64edf751-2ca9-432a-a626-2c23a5f79032/global-pull-secret-syncer/0.log"
Jul 23 01:14:14.127496 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:14.127464    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vgdrk_b4e497b3-17fb-4e64-bd26-b0aec2019720/konnectivity-agent/0.log"
Jul 23 01:14:14.147672 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:14.147640    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-215.ec2.internal_dcbe9759263d750ea2db6eb7b4e9f691/haproxy/0.log"
Jul 23 01:14:19.458288 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:19.458259    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-txt2m_0fa0eb2d-2929-44ba-8cb3-d5b0bf360756/manager/0.log"
Jul 23 01:14:19.658797 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:19.658762    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bfx9c_6e7e0f2c-3558-4054-b460-2da581eb40fd/manager/0.log"
Jul 23 01:14:22.118593 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:22.118557    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wf4m9_a7e819d6-e8d1-47ec-acfc-d58fb660dc66/node-exporter/0.log"
Jul 23 01:14:22.139142 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:22.139117    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wf4m9_a7e819d6-e8d1-47ec-acfc-d58fb660dc66/kube-rbac-proxy/0.log"
Jul 23 01:14:22.161561 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:22.161539    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wf4m9_a7e819d6-e8d1-47ec-acfc-d58fb660dc66/init-textfile/0.log"
Jul 23 01:14:23.810124 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810091    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"]
Jul 23 01:14:23.810626 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810451    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" containerName="istio-proxy"
Jul 23 01:14:23.810626 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810467    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" containerName="istio-proxy"
Jul 23 01:14:23.810626 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810479    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00748046-7284-4250-98d2-e8ac0d9a5148" containerName="maas-api"
Jul 23 01:14:23.810626 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810484    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="00748046-7284-4250-98d2-e8ac0d9a5148" containerName="maas-api"
Jul 23 01:14:23.810626 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810491    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerName="istio-proxy"
Jul 23 01:14:23.810626 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810497    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerName="istio-proxy"
Jul 23 01:14:23.810626 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810508    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerName="istio-proxy"
Jul 23 01:14:23.810626 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810513    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerName="istio-proxy"
Jul 23 01:14:23.810626 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810572    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bf1ccf8-ecb4-4188-9e93-2ec511bb55f4" containerName="istio-proxy"
Jul 23 01:14:23.810626 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810581    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="00748046-7284-4250-98d2-e8ac0d9a5148" containerName="maas-api"
Jul 23 01:14:23.810626 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810588    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerName="istio-proxy"
Jul 23 01:14:23.810626 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.810593    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fba2c9a-e730-4f44-8186-5b623e65a745" containerName="istio-proxy"
Jul 23 01:14:23.813691 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.813668    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:23.816143 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.816121    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tpcmt\"/\"kube-root-ca.crt\""
Jul 23 01:14:23.816269 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.816223    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tpcmt\"/\"openshift-service-ca.crt\""
Jul 23 01:14:23.817427 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.817408    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tpcmt\"/\"default-dockercfg-j8gvw\""
Jul 23 01:14:23.824958 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.824938    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"]
Jul 23 01:14:23.877254 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.877234    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-dsmtv_d908e88c-afcd-4b67-97ef-b36595506a36/networking-console-plugin/0.log"
Jul 23 01:14:23.988067 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.988032    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2349ae3b-5d8b-4c28-abe9-4f02599cd993-proc\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:23.988067 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.988066    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2349ae3b-5d8b-4c28-abe9-4f02599cd993-lib-modules\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:23.988268 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.988096    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2349ae3b-5d8b-4c28-abe9-4f02599cd993-sys\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:23.988268 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.988159    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw42x\" (UniqueName: \"kubernetes.io/projected/2349ae3b-5d8b-4c28-abe9-4f02599cd993-kube-api-access-qw42x\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:23.988268 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:23.988211    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2349ae3b-5d8b-4c28-abe9-4f02599cd993-podres\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.088916 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.088837    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw42x\" (UniqueName: \"kubernetes.io/projected/2349ae3b-5d8b-4c28-abe9-4f02599cd993-kube-api-access-qw42x\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.088916 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.088897    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2349ae3b-5d8b-4c28-abe9-4f02599cd993-podres\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.089067 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.088934    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2349ae3b-5d8b-4c28-abe9-4f02599cd993-proc\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.089067 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.088948    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2349ae3b-5d8b-4c28-abe9-4f02599cd993-lib-modules\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.089067 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.088980    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2349ae3b-5d8b-4c28-abe9-4f02599cd993-sys\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.089067 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.089060    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2349ae3b-5d8b-4c28-abe9-4f02599cd993-sys\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.089221 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.089060    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2349ae3b-5d8b-4c28-abe9-4f02599cd993-proc\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.089221 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.089085    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2349ae3b-5d8b-4c28-abe9-4f02599cd993-podres\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.089221 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.089100    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2349ae3b-5d8b-4c28-abe9-4f02599cd993-lib-modules\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.098637 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.098618    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw42x\" (UniqueName: \"kubernetes.io/projected/2349ae3b-5d8b-4c28-abe9-4f02599cd993-kube-api-access-qw42x\") pod \"perf-node-gather-daemonset-q7wtt\" (UID: \"2349ae3b-5d8b-4c28-abe9-4f02599cd993\") " pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.123823 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.123796    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.250390 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.250349    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"]
Jul 23 01:14:24.252359 ip-10-0-133-215 kubenswrapper[2561]: W0723 01:14:24.252326    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2349ae3b_5d8b_4c28_abe9_4f02599cd993.slice/crio-f9247975dcaf9bc15546fa78aabb3c6138e943fbab7d5edee3732e67c4abd9f4 WatchSource:0}: Error finding container f9247975dcaf9bc15546fa78aabb3c6138e943fbab7d5edee3732e67c4abd9f4: Status 404 returned error can't find the container with id f9247975dcaf9bc15546fa78aabb3c6138e943fbab7d5edee3732e67c4abd9f4
Jul 23 01:14:24.367095 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.367012    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt" event={"ID":"2349ae3b-5d8b-4c28-abe9-4f02599cd993","Type":"ContainerStarted","Data":"64345bbc5e5900ee4bc69dd7cd5e934c16128b34f256cc9682042fe46a1eee4d"}
Jul 23 01:14:24.367095 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.367061    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt" event={"ID":"2349ae3b-5d8b-4c28-abe9-4f02599cd993","Type":"ContainerStarted","Data":"f9247975dcaf9bc15546fa78aabb3c6138e943fbab7d5edee3732e67c4abd9f4"}
Jul 23 01:14:24.367299 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.367105    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:24.383717 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:24.383671    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt" podStartSLOduration=1.383656046 podStartE2EDuration="1.383656046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 01:14:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:14:24.38194468 +0000 UTC m=+3068.188355913" watchObservedRunningTime="2026-07-23 01:14:24.383656046 +0000 UTC m=+3068.190067278"
Jul 23 01:14:26.756411 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:26.756350    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jgpkp_4fbb34aa-fae2-48b7-bda5-57e0031af622/dns/0.log"
Jul 23 01:14:26.779359 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:26.779334    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jgpkp_4fbb34aa-fae2-48b7-bda5-57e0031af622/kube-rbac-proxy/0.log"
Jul 23 01:14:26.803304 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:26.803278    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5z66q_4fdfd2d4-f612-4ee0-b4cd-66417c138705/dns-node-resolver/0.log"
Jul 23 01:14:27.451303 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:27.451244    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hsf6v_410e2099-4588-45d5-a93c-cb4066032492/node-ca/0.log"
Jul 23 01:14:28.517338 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:28.517304    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-g2w67_760e166b-3e6c-47c6-b2f6-3ab7b4180f53/discovery/0.log"
Jul 23 01:14:28.536907 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:28.536876    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-678df7485d-fgqcj_360bf7a3-397c-47be-b280-da4c534f26dd/kube-auth-proxy/0.log"
Jul 23 01:14:28.656662 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:28.656635    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zzxjj_6ded7055-6424-4dd9-8968-b6f3e7ecad22/payload-pre-processing/0.log"
Jul 23 01:14:28.684654 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:28.684625    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-2z2g7_5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904/payload-processing/0.log"
Jul 23 01:14:29.418532 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:29.418483    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xn4kv_197306dc-60be-49d8-854c-dba425eeb32c/serve-healthcheck-canary/0.log"
Jul 23 01:14:29.971759 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:29.971715    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hc78w_f0d486fc-9ffd-46f0-bbc4-044247168063/kube-rbac-proxy/0.log"
Jul 23 01:14:29.991471 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:29.991449    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hc78w_f0d486fc-9ffd-46f0-bbc4-044247168063/exporter/0.log"
Jul 23 01:14:30.011217 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:30.011194    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hc78w_f0d486fc-9ffd-46f0-bbc4-044247168063/extractor/0.log"
Jul 23 01:14:30.381688 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:30.381609    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tpcmt/perf-node-gather-daemonset-q7wtt"
Jul 23 01:14:32.163900 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:32.163874    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kh6dp_816a95a0-4fa8-48ff-9486-45ac658ee1f2/manager/0.log"
Jul 23 01:14:32.358857 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:32.358799    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rnpwl_c5f8583e-370b-4998-a270-4e09f3bb2722/manager/0.log"
Jul 23 01:14:32.484740 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:32.484699    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-795c797fc6-c5v62_b94b6c9a-67da-4989-a5d3-131b1b3352e3/manager/0.log"
Jul 23 01:14:32.511801 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:32.511762    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qqlgk_bc9b2386-9099-45b6-bc22-bbe13ae267c8/server/0.log"
Jul 23 01:14:32.705421 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:32.705395    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-695645577-bn7qh_16466a8f-10c0-4a8e-a961-221d79f2b668/manager/0.log"
Jul 23 01:14:32.735688 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:32.735616    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-695645577-bn7qh_16466a8f-10c0-4a8e-a961-221d79f2b668/copy-manifests/0.log"
Jul 23 01:14:34.291346 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:34.291315    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-cf999_10162be2-0f9c-45f9-a3a9-8dfc01e2e241/openshift-lws-operator/0.log"
Jul 23 01:14:34.543444 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:34.543328    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-g2w67_760e166b-3e6c-47c6-b2f6-3ab7b4180f53/discovery/0.log"
Jul 23 01:14:34.552727 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:34.552704    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-678df7485d-fgqcj_360bf7a3-397c-47be-b280-da4c534f26dd/kube-auth-proxy/0.log"
Jul 23 01:14:34.653971 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:34.653931    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zzxjj_6ded7055-6424-4dd9-8968-b6f3e7ecad22/payload-pre-processing/0.log"
Jul 23 01:14:34.675054 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:34.675028    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-2z2g7_5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904/payload-processing/0.log"
Jul 23 01:14:36.004261 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:36.004228    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-g2w67_760e166b-3e6c-47c6-b2f6-3ab7b4180f53/discovery/0.log"
Jul 23 01:14:36.015050 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:36.015027    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-678df7485d-fgqcj_360bf7a3-397c-47be-b280-da4c534f26dd/kube-auth-proxy/0.log"
Jul 23 01:14:36.127557 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:36.127506    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-zzxjj_6ded7055-6424-4dd9-8968-b6f3e7ecad22/payload-pre-processing/0.log"
Jul 23 01:14:36.148470 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:36.148445    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-2z2g7_5e6ac6bb-d0f3-4cd7-8b7d-9ab2e2cd1904/payload-processing/0.log"
Jul 23 01:14:40.779254 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:40.779223    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b76l8_cd2bec1a-032b-42cd-9555-7154268c805c/kube-multus-additional-cni-plugins/0.log"
Jul 23 01:14:40.799490 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:40.799469    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b76l8_cd2bec1a-032b-42cd-9555-7154268c805c/egress-router-binary-copy/0.log"
Jul 23 01:14:40.821231 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:40.821210    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b76l8_cd2bec1a-032b-42cd-9555-7154268c805c/cni-plugins/0.log"
Jul 23 01:14:40.840136 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:40.840115    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b76l8_cd2bec1a-032b-42cd-9555-7154268c805c/bond-cni-plugin/0.log"
Jul 23 01:14:40.859867 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:40.859842    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b76l8_cd2bec1a-032b-42cd-9555-7154268c805c/routeoverride-cni/0.log"
Jul 23 01:14:40.879934 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:40.879910    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b76l8_cd2bec1a-032b-42cd-9555-7154268c805c/whereabouts-cni-bincopy/0.log"
Jul 23 01:14:40.902339 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:40.902320    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b76l8_cd2bec1a-032b-42cd-9555-7154268c805c/whereabouts-cni/0.log"
Jul 23 01:14:41.124121 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:41.124043    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zgm69_db83e8c2-912c-42dc-8397-b428ebca65b8/kube-multus/0.log"
Jul 23 01:14:41.144510 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:41.144482    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6dn2c_7c65b730-bf89-4090-b8a7-cad9a31bf810/network-metrics-daemon/0.log"
Jul 23 01:14:41.163819 ip-10-0-133-215 kubenswrapper[2561]: I0723 01:14:41.163799    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6dn2c_7c65b730-bf89-4090-b8a7-cad9a31bf810/kube-rbac-proxy/0.log"